Sep 19 01:12:23.684841 ip-10-0-131-76 systemd[1]: Starting Kubernetes Kubelet...
Sep 19 01:12:26.907174 ip-10-0-131-76 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 01:12:26.907174 ip-10-0-131-76 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 19 01:12:26.907174 ip-10-0-131-76 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 01:12:26.907174 ip-10-0-131-76 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 19 01:12:26.907174 ip-10-0-131-76 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 01:12:26.907874 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.907257    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 19 01:12:26.910387 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910371    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 01:12:26.910387 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910386    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 01:12:26.910457 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910390    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 01:12:26.910457 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910393    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 01:12:26.910457 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910396    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 01:12:26.910457 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910400    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 01:12:26.910457 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910403    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 01:12:26.910457 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910406    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 01:12:26.910457 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910408    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 01:12:26.910457 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910413    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 01:12:26.910457 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910417    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 01:12:26.910457 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910421    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 01:12:26.910457 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910424    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 01:12:26.910457 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910426    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 01:12:26.910457 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910429    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 01:12:26.910457 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910432    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 01:12:26.910457 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910434    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 01:12:26.910457 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910437    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 01:12:26.910457 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910439    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 01:12:26.910457 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910442    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 01:12:26.910457 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910448    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 01:12:26.910925 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910451    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 01:12:26.910925 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910453    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 01:12:26.910925 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910456    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 01:12:26.910925 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910458    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 01:12:26.910925 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910461    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 01:12:26.910925 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910465    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 01:12:26.910925 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910467    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 01:12:26.910925 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910470    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 01:12:26.910925 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910473    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 01:12:26.910925 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910476    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 01:12:26.910925 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910478    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 01:12:26.910925 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910489    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 01:12:26.910925 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910505    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 01:12:26.910925 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910509    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 01:12:26.910925 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910512    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 01:12:26.910925 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910516    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 01:12:26.910925 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910519    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 01:12:26.910925 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910521    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 01:12:26.910925 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910524    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 01:12:26.911382 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910526    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 01:12:26.911382 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910529    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 01:12:26.911382 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910531    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 01:12:26.911382 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910534    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 01:12:26.911382 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910536    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 01:12:26.911382 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910539    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 01:12:26.911382 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910541    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 01:12:26.911382 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910544    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 01:12:26.911382 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910546    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 01:12:26.911382 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910549    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 01:12:26.911382 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910552    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 01:12:26.911382 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910554    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 01:12:26.911382 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910557    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 01:12:26.911382 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910560    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 01:12:26.911382 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910563    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 01:12:26.911382 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910566    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 01:12:26.911382 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910569    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 01:12:26.911382 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910571    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 01:12:26.911382 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910574    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 01:12:26.911382 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910577    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 01:12:26.911927 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910580    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 01:12:26.911927 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910584    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 01:12:26.911927 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910587    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 01:12:26.911927 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910590    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 01:12:26.911927 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910592    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 01:12:26.911927 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910595    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 01:12:26.911927 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910598    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 01:12:26.911927 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910600    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 01:12:26.911927 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910605    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 01:12:26.911927 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910607    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 01:12:26.911927 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910610    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 01:12:26.911927 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910612    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 01:12:26.911927 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910615    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 01:12:26.911927 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910617    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 01:12:26.911927 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910620    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 01:12:26.911927 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910622    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 01:12:26.911927 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910625    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 01:12:26.911927 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910627    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 01:12:26.911927 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910630    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 01:12:26.912399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910632    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 01:12:26.912399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910635    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 01:12:26.912399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910637    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 01:12:26.912399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910641    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 01:12:26.912399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910644    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 01:12:26.912399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910646    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 01:12:26.912399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910649    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 01:12:26.912399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910653    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 01:12:26.912399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910656    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 01:12:26.912399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910659    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 01:12:26.912399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910662    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 01:12:26.912399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910664    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 01:12:26.912399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910667    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 01:12:26.912399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910669    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 01:12:26.912399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910672    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 01:12:26.912399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910675    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 01:12:26.912399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910677    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 01:12:26.912399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910680    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 01:12:26.912399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910682    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 01:12:26.912399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.910685    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 01:12:26.912908 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911359    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 01:12:26.912908 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911365    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 01:12:26.912908 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911368    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 01:12:26.912908 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911371    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 01:12:26.912908 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911373    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 01:12:26.912908 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911376    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 01:12:26.912908 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911379    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 01:12:26.912908 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911381    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 01:12:26.912908 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911384    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 01:12:26.912908 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911387    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 01:12:26.912908 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911390    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 01:12:26.912908 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911393    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 01:12:26.912908 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911396    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 01:12:26.912908 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911398    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 01:12:26.912908 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911401    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 01:12:26.912908 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911404    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 01:12:26.912908 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911407    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 01:12:26.912908 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911409    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 01:12:26.912908 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911412    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 01:12:26.912908 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911415    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 01:12:26.913412 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911418    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 01:12:26.913412 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911420    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 01:12:26.913412 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911424    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 01:12:26.913412 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911426    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 01:12:26.913412 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911429    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 01:12:26.913412 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911431    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 01:12:26.913412 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911434    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 01:12:26.913412 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911437    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 01:12:26.913412 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911440    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 01:12:26.913412 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911442    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 01:12:26.913412 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911445    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 01:12:26.913412 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911448    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 01:12:26.913412 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911450    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 01:12:26.913412 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911453    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 01:12:26.913412 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911455    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 01:12:26.913412 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911458    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 01:12:26.913412 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911461    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 01:12:26.913412 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911463    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 01:12:26.913412 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911466    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 01:12:26.913412 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911469    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 01:12:26.913922 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911471    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 01:12:26.913922 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911474    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 01:12:26.913922 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911476    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 01:12:26.913922 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911479    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 01:12:26.913922 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911481    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 01:12:26.913922 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911484    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 01:12:26.913922 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911487    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 01:12:26.913922 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911491    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 01:12:26.913922 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911509    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 01:12:26.913922 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911512    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 01:12:26.913922 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911514    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 01:12:26.913922 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911517    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 01:12:26.913922 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911520    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 01:12:26.913922 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911523    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 01:12:26.913922 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911526    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 01:12:26.913922 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911528    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 01:12:26.913922 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911531    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 01:12:26.913922 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911534    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 01:12:26.913922 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911536    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 01:12:26.913922 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911539    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 01:12:26.914413 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911542    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 01:12:26.914413 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911544    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 01:12:26.914413 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911547    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 01:12:26.914413 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911549    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 01:12:26.914413 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911552    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 01:12:26.914413 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911555    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 01:12:26.914413 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911559    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 01:12:26.914413 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911564    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 01:12:26.914413 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911566    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 01:12:26.914413 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911570    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 01:12:26.914413 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911574    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 01:12:26.914413 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911578    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 01:12:26.914413 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911581    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 01:12:26.914413 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911583    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 01:12:26.914413 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911588    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 01:12:26.914413 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911592    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 01:12:26.914413 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911595    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 01:12:26.914413 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911599    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 01:12:26.914858 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911602    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 01:12:26.914858 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911605    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 01:12:26.914858 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911608    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 01:12:26.914858 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911611    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 01:12:26.914858 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911616    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 01:12:26.914858 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911619    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 01:12:26.914858 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911622    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 01:12:26.914858 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911624    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 01:12:26.914858 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911627    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 01:12:26.914858 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911630    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 01:12:26.914858 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911633    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 01:12:26.914858 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911635    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 01:12:26.914858 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911637    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 01:12:26.914858 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911640    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 01:12:26.914858 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911643    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 01:12:26.914858 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911645    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 01:12:26.914858 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911648    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 01:12:26.914858 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911650    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 01:12:26.914858 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911653    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 01:12:26.914858 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911655    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 01:12:26.915358 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.911658    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 01:12:26.915358 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912009    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 01:12:26.915358 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912014    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 01:12:26.915358 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912017    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 01:12:26.915358 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912020    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 01:12:26.915358 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912022    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 01:12:26.915358 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912025    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 01:12:26.915358 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912028    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 01:12:26.915358 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912030    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 01:12:26.915358 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912033    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 01:12:26.915358 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912035    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 01:12:26.915358 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912038    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 01:12:26.915358 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912040    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 01:12:26.915358 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912043    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 01:12:26.915358 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912046    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 01:12:26.915358 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912048    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 01:12:26.915358 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912053    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 01:12:26.915358 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912056    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 01:12:26.915358 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912059    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 01:12:26.915839 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912061    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 01:12:26.915839 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912065    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 01:12:26.915839 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912068    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 01:12:26.915839 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912071    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 01:12:26.915839 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912073    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 01:12:26.915839 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912076    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 01:12:26.915839 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912078    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 01:12:26.915839 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912081    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 01:12:26.915839 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912083    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 01:12:26.915839 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912086    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 01:12:26.915839 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912089    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 01:12:26.915839 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912091    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 01:12:26.915839 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912094    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 01:12:26.915839 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912098    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 01:12:26.915839 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912102    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 01:12:26.915839 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912105    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 01:12:26.915839 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912107    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 01:12:26.915839 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912110    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 01:12:26.915839 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912113    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 01:12:26.916340 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912115    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 01:12:26.916340 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912126    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 01:12:26.916340 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912129    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 01:12:26.916340 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912132    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 01:12:26.916340 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912135    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 01:12:26.916340 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912137    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 01:12:26.916340 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912140    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 01:12:26.916340 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912143    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 01:12:26.916340 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912146    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 01:12:26.916340 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912148    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 01:12:26.916340 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912153    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 01:12:26.916340 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912155    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 01:12:26.916340 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912158    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 01:12:26.916340 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912160    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 01:12:26.916340 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912163    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 01:12:26.916340 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912166    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 01:12:26.916340 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912169    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 01:12:26.916340 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912171    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 01:12:26.916340 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912174    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 01:12:26.916340 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912176    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 01:12:26.916849 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912179    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 01:12:26.916849 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912182    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 01:12:26.916849 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912184    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 01:12:26.916849 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912187    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 01:12:26.916849 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912189    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 01:12:26.916849 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912192    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 01:12:26.916849 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912194    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 01:12:26.916849 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912197    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 01:12:26.916849 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912199    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 01:12:26.916849 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912202    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 01:12:26.916849 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912205    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 01:12:26.916849 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912207    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 01:12:26.916849 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912210    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 01:12:26.916849 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912213    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 01:12:26.916849 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912216    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 01:12:26.916849 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912218    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 01:12:26.916849 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912221    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 01:12:26.916849 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912223    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 01:12:26.916849 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912226    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 01:12:26.916849 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912228    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 01:12:26.916849 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912231    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 01:12:26.917357 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912233    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 01:12:26.917357 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912238    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 01:12:26.917357 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912240    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 01:12:26.917357 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912243    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 01:12:26.917357 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912245    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 01:12:26.917357 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912248    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 01:12:26.917357 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912250    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 01:12:26.917357 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912253    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 01:12:26.917357 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912256    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 01:12:26.917357 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912258    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 01:12:26.917357 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912261    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 01:12:26.917357 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912264    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 01:12:26.917357 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912268    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 01:12:26.917357 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912271    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 01:12:26.917357 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912274    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 01:12:26.917357 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912276    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 01:12:26.917357 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912279    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 01:12:26.917357 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912282    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 01:12:26.917357 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912284    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 01:12:26.917860 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912288    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 01:12:26.917860 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.912291    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 01:12:26.917860 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912361    2580 flags.go:64] FLAG: --address="0.0.0.0"
Sep 19 01:12:26.917860 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912393    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 19 01:12:26.917860 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912400    2580 flags.go:64] FLAG: --anonymous-auth="true"
Sep 19 01:12:26.917860 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912405    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 19 01:12:26.917860 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912409    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 19 01:12:26.917860 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912413    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 19 01:12:26.917860 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912417    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 19 01:12:26.917860 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912421    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 19 01:12:26.917860 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912424    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 19 01:12:26.917860 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912427    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 19 01:12:26.917860 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912430    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 19 01:12:26.917860 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912434    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 19 01:12:26.917860 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912438    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 19 01:12:26.917860 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912441    2580 flags.go:64] FLAG: --cgroup-root=""
Sep 19 01:12:26.917860 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912444    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 19 01:12:26.917860 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912447    2580 flags.go:64] FLAG: --client-ca-file=""
Sep 19 01:12:26.917860 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912450    2580 flags.go:64] FLAG: --cloud-provider="external"
Sep 19 01:12:26.917860 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912453    2580 flags.go:64] FLAG: --cluster-dns="[]"
Sep 19 01:12:26.917860 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912457    2580 flags.go:64] FLAG: --cluster-domain=""
Sep 19 01:12:26.917860 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912461    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 19 01:12:26.917860 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912464    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 19 01:12:26.918420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912467    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 19 01:12:26.918420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912471    2580 flags.go:64] FLAG: --container-log-max-files="5"
Sep 19 01:12:26.918420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912475    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 19 01:12:26.918420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912478    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 19 01:12:26.918420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912481    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 19 01:12:26.918420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912485    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 19 01:12:26.918420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912488    2580 flags.go:64] FLAG: --contention-profiling="false"
Sep 19 01:12:26.918420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912491    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 19 01:12:26.918420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912508    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 19 01:12:26.918420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912512    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 19 01:12:26.918420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912515    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 19 01:12:26.918420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912519    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 19 01:12:26.918420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912522    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 19 01:12:26.918420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912525    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 19 01:12:26.918420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912528    2580 flags.go:64] FLAG: --enable-load-reader="false"
Sep 19 01:12:26.918420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912531    2580 flags.go:64] FLAG: --enable-server="true"
Sep 19 01:12:26.918420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912534    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 19 01:12:26.918420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912540    2580 flags.go:64] FLAG: --event-burst="100"
Sep 19 01:12:26.918420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912544    2580 flags.go:64] FLAG: --event-qps="50"
Sep 19 01:12:26.918420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912547    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 19 01:12:26.918420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912550    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 19 01:12:26.918420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912553    2580 flags.go:64] FLAG: --eviction-hard=""
Sep 19 01:12:26.918420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912557    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 19 01:12:26.918420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912561    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 19 01:12:26.919004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912564    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 19 01:12:26.919004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912567    2580 flags.go:64] FLAG: --eviction-soft=""
Sep 19 01:12:26.919004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912570    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 19 01:12:26.919004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912573    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 19 01:12:26.919004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912576    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 19 01:12:26.919004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912579    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 19 01:12:26.919004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912582    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 19 01:12:26.919004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912585    2580 flags.go:64] FLAG: --fail-swap-on="true"
Sep 19 01:12:26.919004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912588    2580 flags.go:64] FLAG: --feature-gates=""
Sep 19 01:12:26.919004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912592    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 19 01:12:26.919004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912595    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 19 01:12:26.919004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912598    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 19 01:12:26.919004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912601    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 19 01:12:26.919004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912604    2580 flags.go:64] FLAG: --healthz-port="10248"
Sep 19 01:12:26.919004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912608    2580 flags.go:64] FLAG: --help="false"
Sep 19 01:12:26.919004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912610    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-131-76.ec2.internal"
Sep 19 01:12:26.919004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912614    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 19 01:12:26.919004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912617    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 19 01:12:26.919004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912620    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 19 01:12:26.919004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912623    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 19 01:12:26.919004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912626    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 19 01:12:26.919004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912629    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 19 01:12:26.919004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912632    2580 flags.go:64] FLAG: --image-service-endpoint=""
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912635    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912638    2580 flags.go:64] FLAG: --kube-api-burst="100"
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912641    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912644    2580 flags.go:64] FLAG: --kube-api-qps="50"
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912647    2580 flags.go:64] FLAG: --kube-reserved=""
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912650    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912653    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912656    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912660    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912663    2580 flags.go:64] FLAG: --lock-file=""
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912665    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912668    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912671    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912677    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912680    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912683    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912685    2580 flags.go:64] FLAG: --logging-format="text"
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912689    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912692    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912695    2580 flags.go:64] FLAG: --manifest-url=""
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912699    2580 flags.go:64] FLAG: --manifest-url-header=""
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912704    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912707    2580 flags.go:64] FLAG: --max-open-files="1000000"
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912711    2580 flags.go:64] FLAG: --max-pods="110"
Sep 19 01:12:26.919623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912714    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 19 01:12:26.920265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912717    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 19 01:12:26.920265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912720    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 19 01:12:26.920265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912723    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 19 01:12:26.920265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912726    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 19 01:12:26.920265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912729    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 19 01:12:26.920265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912732    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 19 01:12:26.920265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912739    2580 flags.go:64] FLAG: --node-status-max-images="50"
Sep 19 01:12:26.920265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912743    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 19 01:12:26.920265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912746    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 19 01:12:26.920265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912749    2580 flags.go:64] FLAG: --pod-cidr=""
Sep 19 01:12:26.920265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912752    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 19 01:12:26.920265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912757    2580 flags.go:64] FLAG: --pod-manifest-path=""
Sep 19 01:12:26.920265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912760    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 19 01:12:26.920265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912764    2580 flags.go:64] FLAG: --pods-per-core="0"
Sep 19 01:12:26.920265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912769    2580 flags.go:64] FLAG: --port="10250"
Sep 19 01:12:26.920265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912775    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 19 01:12:26.920265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912779    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0bbf336aab2f0339a"
Sep 19 01:12:26.920265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912784    2580 flags.go:64] FLAG: --qos-reserved=""
Sep 19 01:12:26.920265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912788    2580 flags.go:64] FLAG: --read-only-port="10255"
Sep 19 01:12:26.920265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912793    2580 flags.go:64] FLAG: --register-node="true"
Sep 19 01:12:26.920265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912797    2580 flags.go:64] FLAG: --register-with-taints=""
Sep 19 01:12:26.920265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912802    2580 flags.go:64] FLAG: --registry-burst="10"
Sep 19 01:12:26.920265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912805    2580 flags.go:64] FLAG: --registry-qps="5"
Sep 19 01:12:26.920265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912808    2580 flags.go:64] FLAG: --reserved-cpus=""
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912811    2580 flags.go:64] FLAG: --reserved-memory=""
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912821    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912824    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912827    2580 flags.go:64] FLAG: --rotate-certificates="false"
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912832    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912835    2580 flags.go:64] FLAG: --runonce="false"
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912838    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912841    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912844    2580 flags.go:64] FLAG: --seccomp-default="false"
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912847    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912850    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912853    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912856    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912860    2580 flags.go:64] FLAG: --storage-driver-password="root"
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912863    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912866    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912868    2580 flags.go:64] FLAG: --storage-driver-user="root"
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912871    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912874    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912877    2580 flags.go:64] FLAG: --system-cgroups=""
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912880    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912886    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912888    2580 flags.go:64] FLAG: --tls-cert-file=""
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912892    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 19 01:12:26.921189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912897    2580 flags.go:64] FLAG: --tls-min-version=""
Sep 19 01:12:26.921945 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912899    2580 flags.go:64] FLAG: --tls-private-key-file=""
Sep 19 01:12:26.921945 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912902    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 19 01:12:26.921945 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912905    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 19 01:12:26.921945 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912908    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 19 01:12:26.921945 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912912    2580 flags.go:64] FLAG: --v="2"
Sep 19 01:12:26.921945 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912916    2580 flags.go:64] FLAG: --version="false"
Sep 19 01:12:26.921945 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912920    2580 flags.go:64] FLAG: --vmodule=""
Sep 19 01:12:26.921945 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912924    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 19 01:12:26.921945 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.912927    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 19 01:12:26.921945 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913042    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 01:12:26.921945 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913046    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 01:12:26.921945 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913051    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 01:12:26.921945 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913053    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 01:12:26.921945 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913057    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 01:12:26.921945 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913059    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 01:12:26.921945 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913062    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 01:12:26.921945 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913065    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 01:12:26.921945 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913068    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 01:12:26.921945 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913070    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 01:12:26.921945 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913073    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 01:12:26.921945 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913076    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 01:12:26.921945 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913078    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 01:12:26.922487 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913089    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 01:12:26.922487 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913092    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 01:12:26.922487 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913094    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 01:12:26.922487 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913097    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 01:12:26.922487 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913100    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 01:12:26.922487 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913102    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 01:12:26.922487 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913105    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 01:12:26.922487 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913107    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 01:12:26.922487 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913111    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 01:12:26.922487 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913114    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 01:12:26.922487 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913116    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 01:12:26.922487 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913119    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 01:12:26.922487 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913122    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 01:12:26.922487 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913125    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 01:12:26.922487 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913127    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 01:12:26.922487 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913130    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 01:12:26.922487 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913133    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 01:12:26.922487 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913135    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 01:12:26.922487 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913138    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 01:12:26.922487 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913140    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 01:12:26.923016 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913143    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 01:12:26.923016 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913148    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 01:12:26.923016 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913150    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 01:12:26.923016 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913153    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 01:12:26.923016 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913156    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 01:12:26.923016 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913158    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 01:12:26.923016 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913161    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 01:12:26.923016 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913164    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 01:12:26.923016 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913166    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 01:12:26.923016 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913171    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 01:12:26.923016 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913174    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 01:12:26.923016 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913177    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 01:12:26.923016 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913180    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 01:12:26.923016 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913183    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 01:12:26.923016 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913186    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 01:12:26.923016 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913189    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 01:12:26.923016 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913192    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 01:12:26.923016 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913194    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 01:12:26.923016 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913197    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 01:12:26.923486 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913200    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 01:12:26.923486 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913204    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 01:12:26.923486 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913207    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 01:12:26.923486 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913210    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 01:12:26.923486 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913212    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 01:12:26.923486 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913215    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 01:12:26.923486 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913218    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 01:12:26.923486 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913221    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 01:12:26.923486 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913223    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 01:12:26.923486 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913226    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 01:12:26.923486 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913228    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 01:12:26.923486 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913231    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 01:12:26.923486 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913233    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 01:12:26.923486 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913236    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 01:12:26.923486 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913240    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 01:12:26.923486 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913243    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 01:12:26.923486 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913245    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 01:12:26.923486 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913248    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 01:12:26.923486 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913251    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 01:12:26.923979 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913255    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 01:12:26.923979 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913258    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 01:12:26.923979 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913260    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 01:12:26.923979 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913263    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 01:12:26.923979 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913266    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 01:12:26.923979 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913268    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 01:12:26.923979 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913271    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 01:12:26.923979 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913274    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 01:12:26.923979 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913276    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 01:12:26.923979 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913279    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 01:12:26.923979 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913281    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 01:12:26.923979 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913284    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 01:12:26.923979 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913286    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 01:12:26.923979 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913289    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 01:12:26.923979 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913292    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 01:12:26.923979 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913294    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 01:12:26.923979 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913297    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 01:12:26.923979 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913299    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 01:12:26.923979 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913302    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 01:12:26.923979 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913304    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 01:12:26.924477 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913307    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 01:12:26.924477 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913309    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 01:12:26.924477 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913312    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 01:12:26.924477 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913315    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 01:12:26.924477 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913317    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 01:12:26.924477 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913320    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 01:12:26.924477 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913322    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 01:12:26.924477 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.913326    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 01:12:26.924477 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.913951    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 01:12:26.924477 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.922143    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 19 01:12:26.924477 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.922158    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 19 01:12:26.924477 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.922177    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 19 01:12:26.924477 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.922183    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 19 01:12:26.924477 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922240    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 01:12:26.924477 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922246    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 01:12:26.924868 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922249    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 01:12:26.924868 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922253    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 01:12:26.924868 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922256    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 01:12:26.924868 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922259    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 01:12:26.924868 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922262    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 01:12:26.924868 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922265    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 01:12:26.924868 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922268    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 01:12:26.924868 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922271    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 01:12:26.924868 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922275    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 01:12:26.924868 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922278    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 01:12:26.924868 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922280    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 01:12:26.924868 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922284    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 01:12:26.924868 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922287    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 01:12:26.924868 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922290    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 01:12:26.924868 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922292    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 01:12:26.924868 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922295    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 01:12:26.924868 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922297    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 01:12:26.924868 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922300    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 01:12:26.924868 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922303    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 01:12:26.924868 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922305    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 01:12:26.925399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922308    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 01:12:26.925399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922310    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 01:12:26.925399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922314    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 01:12:26.925399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922317    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 01:12:26.925399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922319    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 01:12:26.925399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922322    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 01:12:26.925399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922325    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 01:12:26.925399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922329    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 01:12:26.925399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922331    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 01:12:26.925399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922334    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 01:12:26.925399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922337    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 01:12:26.925399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922339    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 01:12:26.925399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922342    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 01:12:26.925399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922345    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 01:12:26.925399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922347    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 01:12:26.925399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922350    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 01:12:26.925399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922353    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 01:12:26.925399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922355    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 01:12:26.925399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922358    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 01:12:26.925399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922360    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 01:12:26.925399 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922363    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 01:12:26.925928 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922366    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 01:12:26.925928 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922368    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 01:12:26.925928 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922371    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 01:12:26.925928 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922374    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 01:12:26.925928 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922378    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 01:12:26.925928 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922383    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 01:12:26.925928 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922387    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 01:12:26.925928 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922390    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 01:12:26.925928 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922393    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 01:12:26.925928 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922396    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 01:12:26.925928 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922399    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 01:12:26.925928 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922402    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 01:12:26.925928 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922404    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 01:12:26.925928 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922407    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 01:12:26.925928 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922409    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 01:12:26.925928 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922412    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 01:12:26.925928 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922415    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 01:12:26.925928 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922418    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 01:12:26.925928 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922421    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 01:12:26.926396 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922424    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 01:12:26.926396 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922427    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 01:12:26.926396 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922430    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 01:12:26.926396 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922432    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 01:12:26.926396 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922435    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 01:12:26.926396 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922437    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 01:12:26.926396 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922440    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 01:12:26.926396 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922443    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 01:12:26.926396 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922445    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 01:12:26.926396 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922449    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 01:12:26.926396 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922452    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 01:12:26.926396 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922455    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 01:12:26.926396 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922459    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 01:12:26.926396 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922461    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 01:12:26.926396 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922464    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 01:12:26.926396 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922467    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 01:12:26.926396 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922470    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 01:12:26.926396 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922473    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 01:12:26.926396 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922476    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 01:12:26.926930 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922478    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 01:12:26.926930 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922481    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 01:12:26.926930 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922484    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 01:12:26.926930 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922486    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 01:12:26.926930 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922489    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 01:12:26.926930 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922492    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 01:12:26.926930 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922509    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 01:12:26.926930 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922512    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 01:12:26.926930 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922515    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 01:12:26.926930 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922518    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 01:12:26.926930 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922521    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 01:12:26.926930 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922524    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 01:12:26.926930 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922527    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 01:12:26.926930 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922530    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 01:12:26.926930 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922532    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 01:12:26.926930 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922535    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 01:12:26.926930 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922538    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 01:12:26.926930 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922541    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 01:12:26.927359 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.922546    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 01:12:26.927359 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922662    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 01:12:26.927359 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922667    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 01:12:26.927359 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922669    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 01:12:26.927359 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922673    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 01:12:26.927359 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922676    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 01:12:26.927359 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922680    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 01:12:26.927359 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922684    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 01:12:26.927359 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922687    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 01:12:26.927359 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922689    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 01:12:26.927359 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922692    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 01:12:26.927359 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922695    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 01:12:26.927359 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922697    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 01:12:26.927359 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922700    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 01:12:26.927724 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922703    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 01:12:26.927724 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922705    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 01:12:26.927724 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922708    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 01:12:26.927724 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922711    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 01:12:26.927724 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922714    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 01:12:26.927724 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922716    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 01:12:26.927724 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922719    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 01:12:26.927724 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922721    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 01:12:26.927724 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922724    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 01:12:26.927724 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922727    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 01:12:26.927724 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922730    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 01:12:26.927724 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922732    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 01:12:26.927724 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922735    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 01:12:26.927724 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922738    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 01:12:26.927724 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922740    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 01:12:26.927724 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922743    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 01:12:26.927724 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922745    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 01:12:26.927724 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922749    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 01:12:26.927724 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922751    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 01:12:26.927724 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922754    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 01:12:26.928217 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922757    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 01:12:26.928217 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922759    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 01:12:26.928217 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922762    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 01:12:26.928217 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922765    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 01:12:26.928217 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922769    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 01:12:26.928217 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922771    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 01:12:26.928217 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922774    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 01:12:26.928217 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922777    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 01:12:26.928217 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922779    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 01:12:26.928217 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922782    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 01:12:26.928217 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922785    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 01:12:26.928217 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922787    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 01:12:26.928217 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922791    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 01:12:26.928217 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922794    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 01:12:26.928217 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922797    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 01:12:26.928217 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922799    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 01:12:26.928217 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922802    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 01:12:26.928217 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922804    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 01:12:26.928217 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922807    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 01:12:26.928217 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922810    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 01:12:26.928739 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922812    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 01:12:26.928739 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922815    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 01:12:26.928739 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922818    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 01:12:26.928739 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922820    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 01:12:26.928739 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922823    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 01:12:26.928739 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922825    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 01:12:26.928739 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922827    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 01:12:26.928739 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922830    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 01:12:26.928739 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922833    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 01:12:26.928739 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922835    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 01:12:26.928739 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922838    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 01:12:26.928739 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922841    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 01:12:26.928739 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922844    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 01:12:26.928739 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922847    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 01:12:26.928739 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922850    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 01:12:26.928739 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922852    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 01:12:26.928739 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922855    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 01:12:26.928739 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922857    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 01:12:26.928739 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922860    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 01:12:26.928739 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922863    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 01:12:26.929226 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922865    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 01:12:26.929226 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922868    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 01:12:26.929226 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922870    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 01:12:26.929226 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922874    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 01:12:26.929226 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922878    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 01:12:26.929226 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922882    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 01:12:26.929226 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922885    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 01:12:26.929226 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922888    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 01:12:26.929226 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922891    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 01:12:26.929226 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922893    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 01:12:26.929226 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922896    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 01:12:26.929226 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922899    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 01:12:26.929226 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922902    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 01:12:26.929226 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922905    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 01:12:26.929226 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922908    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 01:12:26.929226 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922910    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 01:12:26.929226 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922913    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 01:12:26.929226 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922916    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 01:12:26.929226 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922919    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 01:12:26.929870 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922922    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 01:12:26.929870 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922924    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 01:12:26.929870 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922927    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 01:12:26.929870 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922929    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 01:12:26.929870 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922932    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 01:12:26.929870 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922935    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 01:12:26.929870 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:26.922937    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 01:12:26.929870 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.922942    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 01:12:26.929870 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.923034    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 19 01:12:26.929870 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.925276    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 19 01:12:26.929870 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.926109    2580 server.go:1019] "Starting client certificate rotation"
Sep 19 01:12:26.929870 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.926208    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 19 01:12:26.929870 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.926248    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 19 01:12:26.930253 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.928401    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 19 01:12:26.930253 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.929480    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 19 01:12:26.942574 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.942559    2580 log.go:25] "Validated CRI v1 runtime API"
Sep 19 01:12:26.944133 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.944105    2580 log.go:25] "Validated CRI v1 image API"
Sep 19 01:12:26.945621 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.945606    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 19 01:12:26.948362 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.947777    2580 fs.go:135] Filesystem UUIDs: map[33592b1c-3d34-4b85-8ced-13c44dd35b18:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a5e6fce0-bc6d-4936-85f5-70ca98877da9:/dev/nvme0n1p4]
Sep 19 01:12:26.948362 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.947799    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 19 01:12:26.953447 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.953338    2580 manager.go:216] Machine: {Timestamp:2026-09-19 01:12:26.952065531 +0000 UTC m=+1.858927569 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102290 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2552c4df72e8182434bdde59888a33 SystemUUID:ec2552c4-df72-e818-2434-bdde59888a33 BootID:2f6e85b7-4609-4008-ab2d-b7abd4a1ad0b Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ed:0d:87:b7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ed:0d:87:b7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f2:ec:1f:61:ae:d6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 19 01:12:26.953447 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.953440    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 19 01:12:26.953578 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.953544    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 19 01:12:26.955226 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.955201    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 19 01:12:26.955400 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.955227    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-76.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 19 01:12:26.956437 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.956427    2580 topology_manager.go:138] "Creating topology manager with none policy"
Sep 19 01:12:26.957018 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.956440    2580 container_manager_linux.go:309] "Creating device plugin manager"
Sep 19 01:12:26.957018 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.956448    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 19 01:12:26.957018 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.956458    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 19 01:12:26.957018 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.956540    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 19 01:12:26.960540 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.960528    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 19 01:12:26.960659 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.960650    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 19 01:12:26.960696 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.960688    2580 kubelet.go:486] "Attempting to sync node with API server"
Sep 19 01:12:26.960734 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.960712    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 19 01:12:26.960734 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.960729    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 19 01:12:26.960803 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.960738    2580 kubelet.go:398] "Adding apiserver pod source"
Sep 19 01:12:26.960803 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.960747    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 19 01:12:26.961176 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.961157    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 19 01:12:26.961176 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.961181    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 19 01:12:26.963619 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.963592    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 19 01:12:26.966170 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.966154    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 19 01:12:26.966249 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.966185    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 19 01:12:26.967341 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.967329    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 19 01:12:26.967403 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.967345    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 19 01:12:26.967403 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.967351    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 19 01:12:26.967403 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.967357    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 19 01:12:26.967403 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.967366    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 19 01:12:26.967403 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.967371    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 19 01:12:26.967403 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.967394    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 19 01:12:26.967403 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.967399    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 19 01:12:26.967403 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.967406    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 19 01:12:26.967652 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.967412    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 19 01:12:26.967652 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.967420    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 19 01:12:26.967652 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.967426    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 19 01:12:26.967652 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.967454    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 19 01:12:26.967652 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.967459    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 19 01:12:26.967652 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.967606    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 19 01:12:26.971213 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.971195    2580 server.go:1268] "Started kubelet"
Sep 19 01:12:26.971291 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.971266    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 19 01:12:26.971328 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.971282    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 19 01:12:26.971367 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.971355    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 19 01:12:26.972155 ip-10-0-131-76 systemd[1]: Started Kubernetes Kubelet.
Sep 19 01:12:26.972793 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.972516    2580 server.go:310] "Adding debug handlers to kubelet server"
Sep 19 01:12:26.972917 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:26.972844    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-76.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 19 01:12:26.972968 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:26.972918    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 19 01:12:26.972968 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.972950    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-76.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 19 01:12:26.975304 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.975276    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 19 01:12:26.975594 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:26.975576    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 19 01:12:26.981137 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.981123    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 19 01:12:26.981137 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.981135    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 19 01:12:26.981230 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:26.981189    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-76.ec2.internal\" not found"
Sep 19 01:12:26.981743 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.981729    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 19 01:12:26.981829 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.981734    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 19 01:12:26.981829 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.981771    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 19 01:12:26.981933 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.981888    2580 reconstruct.go:97] "Volume reconstruction finished"
Sep 19 01:12:26.981933 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.981896    2580 reconciler.go:29] "Reconciler: start to sync state"
Sep 19 01:12:26.981933 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.981904    2580 factory.go:55] Registering systemd factory
Sep 19 01:12:26.981933 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.981920    2580 factory.go:223] Registration of the systemd container factory successfully
Sep 19 01:12:26.982149 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.982133    2580 factory.go:153] Registering CRI-O factory
Sep 19 01:12:26.982149 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.982149    2580 factory.go:223] Registration of the crio container factory successfully
Sep 19 01:12:26.982299 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.982199    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 19 01:12:26.982299 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.982216    2580 factory.go:103] Registering Raw factory
Sep 19 01:12:26.982299 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.982227    2580 manager.go:1196] Started watching for new ooms in manager
Sep 19 01:12:26.982524 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:26.982513    2580 manager.go:349] Starting recovery of all containers
Sep 19 01:12:26.983820 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:26.983798    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 19 01:12:26.983901 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:26.983882    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-76.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 19 01:12:27.002776 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.002756    2580 manager.go:354] Recovery completed
Sep 19 01:12:27.007747 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.007723    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 01:12:27.009237 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.009220    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-76.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 01:12:27.009302 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.009250    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-76.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 01:12:27.009302 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.009261    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-76.ec2.internal" event="NodeHasSufficientPID"
Sep 19 01:12:27.009820 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.009804    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 19 01:12:27.009820 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.009818    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 19 01:12:27.009935 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.009821    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 01:12:27.009935 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.009834    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 19 01:12:27.010460 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.010443    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-76.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 01:12:27.010554 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.010469    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-76.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 01:12:27.010554 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.010478    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-76.ec2.internal" event="NodeHasSufficientPID"
Sep 19 01:12:27.012025 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.012012    2580 policy_none.go:49] "None policy: Start"
Sep 19 01:12:27.012097 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.012029    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 19 01:12:27.012097 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.012041    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 19 01:12:27.013128 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:27.012337    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-76.ec2.internal.18d693be2f311a82  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-76.ec2.internal,UID:ip-10-0-131-76.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-76.ec2.internal,},FirstTimestamp:2026-09-19 01:12:26.971167362 +0000 UTC m=+1.878029402,LastTimestamp:2026-09-19 01:12:26.971167362 +0000 UTC m=+1.878029402,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-76.ec2.internal,}"
Sep 19 01:12:27.014028 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.014016    2580 policy_none.go:47] "Start"
Sep 19 01:12:27.023980 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:27.023921    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-76.ec2.internal.18d693be317602d4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-76.ec2.internal,UID:ip-10-0-131-76.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-76.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-76.ec2.internal,},FirstTimestamp:2026-09-19 01:12:27.009237716 +0000 UTC m=+1.916099755,LastTimestamp:2026-09-19 01:12:27.009237716 +0000 UTC m=+1.916099755,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-76.ec2.internal,}"
Sep 19 01:12:27.024438 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.024421    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7mrvt"
Sep 19 01:12:27.032532 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.032515    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7mrvt"
Sep 19 01:12:27.033116 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:27.033058    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-76.ec2.internal.18d693be317645e1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-76.ec2.internal,UID:ip-10-0-131-76.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-76.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-76.ec2.internal,},FirstTimestamp:2026-09-19 01:12:27.009254881 +0000 UTC m=+1.916116919,LastTimestamp:2026-09-19 01:12:27.009254881 +0000 UTC m=+1.916116919,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-76.ec2.internal,}"
Sep 19 01:12:27.067388 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.045810    2580 manager.go:337] "Starting Device Plugin manager"
Sep 19 01:12:27.067388 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:27.045845    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 19 01:12:27.067388 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.045855    2580 server.go:87] "Starting device plugin registration server"
Sep 19 01:12:27.067388 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.046091    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 19 01:12:27.067388 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.046114    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 19 01:12:27.067388 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.046187    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 19 01:12:27.067388 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.046261    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 19 01:12:27.067388 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.046269    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 19 01:12:27.067388 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:27.046829    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 19 01:12:27.067388 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:27.046877    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-76.ec2.internal\" not found"
Sep 19 01:12:27.067388 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.055089    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 19 01:12:27.067388 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.056224    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 19 01:12:27.067388 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.056262    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 19 01:12:27.067388 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.056279    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 19 01:12:27.067388 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:27.056313    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 19 01:12:27.067388 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.058846    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 01:12:27.146547 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.146525    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 01:12:27.147418 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.147403    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-76.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 01:12:27.147554 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.147434    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-76.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 01:12:27.147554 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.147448    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-76.ec2.internal" event="NodeHasSufficientPID"
Sep 19 01:12:27.147554 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.147473    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-76.ec2.internal"
Sep 19 01:12:27.157043 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.157024    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-131-76.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-76.ec2.internal"]
Sep 19 01:12:27.157139 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.157083    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 01:12:27.157809 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.157760    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-76.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 01:12:27.157809 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.157790    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-76.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 01:12:27.157809 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.157805    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-76.ec2.internal" event="NodeHasSufficientPID"
Sep 19 01:12:27.157987 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.157973    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-76.ec2.internal"
Sep 19 01:12:27.158020 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:27.157993    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-76.ec2.internal\": node \"ip-10-0-131-76.ec2.internal\" not found"
Sep 19 01:12:27.160111 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.160100    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 01:12:27.160239 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.160206    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-76.ec2.internal"
Sep 19 01:12:27.160239 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.160234    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 01:12:27.160812 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.160796    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-76.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 01:12:27.160812 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.160805    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-76.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 01:12:27.160922 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.160824    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-76.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 01:12:27.160922 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.160828    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-76.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 01:12:27.160922 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.160839    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-76.ec2.internal" event="NodeHasSufficientPID"
Sep 19 01:12:27.160922 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.160841    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-76.ec2.internal" event="NodeHasSufficientPID"
Sep 19 01:12:27.163144 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.163129    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-76.ec2.internal"
Sep 19 01:12:27.163208 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.163163    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 01:12:27.163779 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.163762    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-76.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 01:12:27.163875 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.163799    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-76.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 01:12:27.163875 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.163810    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-76.ec2.internal" event="NodeHasSufficientPID"
Sep 19 01:12:27.177025 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:27.177000    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-76.ec2.internal\" not found" node="ip-10-0-131-76.ec2.internal"
Sep 19 01:12:27.180683 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:27.180670    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-76.ec2.internal\" not found" node="ip-10-0-131-76.ec2.internal"
Sep 19 01:12:27.199108 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:27.199088    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-76.ec2.internal\" not found"
Sep 19 01:12:27.283791 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.283768    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0034bf5044f034aee1efd2c2a680ecba-config\") pod \"kube-apiserver-proxy-ip-10-0-131-76.ec2.internal\" (UID: \"0034bf5044f034aee1efd2c2a680ecba\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-76.ec2.internal"
Sep 19 01:12:27.283882 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.283794    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/60e7758b2d5dde524dbb14f374f071d5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-76.ec2.internal\" (UID: \"60e7758b2d5dde524dbb14f374f071d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-76.ec2.internal"
Sep 19 01:12:27.283882 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.283812    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/60e7758b2d5dde524dbb14f374f071d5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-76.ec2.internal\" (UID: \"60e7758b2d5dde524dbb14f374f071d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-76.ec2.internal"
Sep 19 01:12:27.299175 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:27.299156    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-76.ec2.internal\" not found"
Sep 19 01:12:27.384091 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.384064    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0034bf5044f034aee1efd2c2a680ecba-config\") pod \"kube-apiserver-proxy-ip-10-0-131-76.ec2.internal\" (UID: \"0034bf5044f034aee1efd2c2a680ecba\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-76.ec2.internal"
Sep 19 01:12:27.384091 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.384092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/60e7758b2d5dde524dbb14f374f071d5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-76.ec2.internal\" (UID: \"60e7758b2d5dde524dbb14f374f071d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-76.ec2.internal"
Sep 19 01:12:27.384260 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.384109    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/60e7758b2d5dde524dbb14f374f071d5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-76.ec2.internal\" (UID: \"60e7758b2d5dde524dbb14f374f071d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-76.ec2.internal"
Sep 19 01:12:27.384260 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.384139    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/60e7758b2d5dde524dbb14f374f071d5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-76.ec2.internal\" (UID: \"60e7758b2d5dde524dbb14f374f071d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-76.ec2.internal"
Sep 19 01:12:27.384260 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.384165    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0034bf5044f034aee1efd2c2a680ecba-config\") pod \"kube-apiserver-proxy-ip-10-0-131-76.ec2.internal\" (UID: \"0034bf5044f034aee1efd2c2a680ecba\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-76.ec2.internal"
Sep 19 01:12:27.384260 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.384165    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/60e7758b2d5dde524dbb14f374f071d5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-76.ec2.internal\" (UID: \"60e7758b2d5dde524dbb14f374f071d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-76.ec2.internal"
Sep 19 01:12:27.400193 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:27.400165    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-76.ec2.internal\" not found"
Sep 19 01:12:27.481660 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.481611    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-76.ec2.internal"
Sep 19 01:12:27.484932 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.484907    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-76.ec2.internal"
Sep 19 01:12:27.500345 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:27.500218    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-76.ec2.internal\" not found"
Sep 19 01:12:27.600765 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:27.600735    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-76.ec2.internal\" not found"
Sep 19 01:12:27.701202 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:27.701171    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-76.ec2.internal\" not found"
Sep 19 01:12:27.801669 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:27.801620    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-76.ec2.internal\" not found"
Sep 19 01:12:27.902101 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:27.902071    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-76.ec2.internal\" not found"
Sep 19 01:12:27.926251 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.926226    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 19 01:12:27.926642 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:27.926364    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 19 01:12:28.003007 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:28.002980    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-76.ec2.internal\" not found"
Sep 19 01:12:28.027142 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.027114    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 01:12:28.039737 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.039704    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-18 01:07:27 +0000 UTC" deadline="2028-03-25 12:05:02.889601405 +0000 UTC"
Sep 19 01:12:28.039807 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.039737    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13282h52m34.849867667s"
Sep 19 01:12:28.064078 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.064033    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 01:12:28.082018 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.081999    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-76.ec2.internal"
Sep 19 01:12:28.092125 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.092106    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 19 01:12:28.093821 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.093798    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-76.ec2.internal"
Sep 19 01:12:28.106747 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.106732    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 19 01:12:28.171636 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:28.171606    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60e7758b2d5dde524dbb14f374f071d5.slice/crio-e886711f166a6ff5f34ee74ab9cd19eb81abf55017939922404126fecdb26638 WatchSource:0}: Error finding container e886711f166a6ff5f34ee74ab9cd19eb81abf55017939922404126fecdb26638: Status 404 returned error can't find the container with id e886711f166a6ff5f34ee74ab9cd19eb81abf55017939922404126fecdb26638
Sep 19 01:12:28.171998 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:28.171981    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0034bf5044f034aee1efd2c2a680ecba.slice/crio-521891dc7a46fcd739f2187f31a1e174b86508c8089a46346f05914ed40332ea WatchSource:0}: Error finding container 521891dc7a46fcd739f2187f31a1e174b86508c8089a46346f05914ed40332ea: Status 404 returned error can't find the container with id 521891dc7a46fcd739f2187f31a1e174b86508c8089a46346f05914ed40332ea
Sep 19 01:12:28.176696 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.176682    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 01:12:28.482647 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.482560    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 01:12:28.848831 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.848750    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 01:12:28.962411 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.962382    2580 apiserver.go:52] "Watching apiserver"
Sep 19 01:12:28.967929 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.967904    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 19 01:12:28.968436 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.968414    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-kj96s","kube-system/kube-apiserver-proxy-ip-10-0-131-76.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk","openshift-dns/node-resolver-grkk4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-76.ec2.internal","openshift-ovn-kubernetes/ovnkube-node-7z77w","kube-system/konnectivity-agent-9n69k","openshift-cluster-node-tuning-operator/tuned-46sks","openshift-image-registry/node-ca-dvd6m","openshift-multus/multus-additional-cni-plugins-tfq72","openshift-multus/multus-d4nk2","openshift-multus/network-metrics-daemon-hpjz8","openshift-network-diagnostics/network-check-target-nh6vq","openshift-network-operator/iptables-alerter-j6q6r"]
Sep 19 01:12:28.971033 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.971011    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9n69k"
Sep 19 01:12:28.973380 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.973312    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:28.973544 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.973526    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 19 01:12:28.973731 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.973713    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 19 01:12:28.974146 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.973962    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-f5z2m\""
Sep 19 01:12:28.975878 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.975487    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 19 01:12:28.975973 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.975889    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lwkns\""
Sep 19 01:12:28.976213 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.976159    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 19 01:12:28.976352 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.976335    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 19 01:12:28.976879 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.976474    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 19 01:12:28.976879 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.976565    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk"
Sep 19 01:12:28.977676 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.977654    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-grkk4"
Sep 19 01:12:28.978526 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.978488    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 19 01:12:28.978653 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.978545    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vxt45\""
Sep 19 01:12:28.978762 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.978746    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 19 01:12:28.978892 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.978788    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 19 01:12:28.980769 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.979986    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 19 01:12:28.980769 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.980228    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-l27rc\""
Sep 19 01:12:28.980769 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.980572    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 19 01:12:28.982639 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.982616    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 19 01:12:28.986522 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.986490    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j6q6r"
Sep 19 01:12:28.988710 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.988690    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 19 01:12:28.989049 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.988904    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 19 01:12:28.989114 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.989096    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gsx45\""
Sep 19 01:12:28.989727 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.989710    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 19 01:12:28.991148 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.991130    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:28.992049 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992026    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrkx4\" (UniqueName: \"kubernetes.io/projected/3347022a-f11e-4bad-a7dd-902cf5613aa5-kube-api-access-jrkx4\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:28.992151 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992065    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-host-run-multus-certs\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:28.992151 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992090    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/da7b0d9d-cbe8-4f97-87e0-806003cd917b-etc-selinux\") pod \"aws-ebs-csi-driver-node-5wqmk\" (UID: \"da7b0d9d-cbe8-4f97-87e0-806003cd917b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk"
Sep 19 01:12:28.992151 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992114    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/da7b0d9d-cbe8-4f97-87e0-806003cd917b-sys-fs\") pod \"aws-ebs-csi-driver-node-5wqmk\" (UID: \"da7b0d9d-cbe8-4f97-87e0-806003cd917b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk"
Sep 19 01:12:28.992307 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992146    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcbl4\" (UniqueName: \"kubernetes.io/projected/56a6eadc-607c-4b80-8dc8-183e63551d8b-kube-api-access-zcbl4\") pod \"iptables-alerter-j6q6r\" (UID: \"56a6eadc-607c-4b80-8dc8-183e63551d8b\") " pod="openshift-network-operator/iptables-alerter-j6q6r"
Sep 19 01:12:28.992307 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992198    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2f997c70-10b2-49e0-8711-00e914baa6c4-agent-certs\") pod \"konnectivity-agent-9n69k\" (UID: \"2f997c70-10b2-49e0-8711-00e914baa6c4\") " pod="kube-system/konnectivity-agent-9n69k"
Sep 19 01:12:28.992307 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-host-var-lib-cni-bin\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:28.992307 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992248    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3347022a-f11e-4bad-a7dd-902cf5613aa5-multus-daemon-config\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:28.992307 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992264    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 19 01:12:28.992307 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992269    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/da7b0d9d-cbe8-4f97-87e0-806003cd917b-device-dir\") pod \"aws-ebs-csi-driver-node-5wqmk\" (UID: \"da7b0d9d-cbe8-4f97-87e0-806003cd917b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk"
Sep 19 01:12:28.992307 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992291    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/54557d74-f7ed-44ac-b483-9942019a9c4d-hosts-file\") pod \"node-resolver-grkk4\" (UID: \"54557d74-f7ed-44ac-b483-9942019a9c4d\") " pod="openshift-dns/node-resolver-grkk4"
Sep 19 01:12:28.992644 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992312    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/56a6eadc-607c-4b80-8dc8-183e63551d8b-iptables-alerter-script\") pod \"iptables-alerter-j6q6r\" (UID: \"56a6eadc-607c-4b80-8dc8-183e63551d8b\") " pod="openshift-network-operator/iptables-alerter-j6q6r"
Sep 19 01:12:28.992644 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992348    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2f997c70-10b2-49e0-8711-00e914baa6c4-konnectivity-ca\") pod \"konnectivity-agent-9n69k\" (UID: \"2f997c70-10b2-49e0-8711-00e914baa6c4\") " pod="kube-system/konnectivity-agent-9n69k"
Sep 19 01:12:28.992644 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992368    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-system-cni-dir\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:28.992644 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992390    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-cnibin\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:28.992644 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992412    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-multus-socket-dir-parent\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:28.992644 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992448    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-host-var-lib-cni-multus\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:28.992644 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992568    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-host-var-lib-kubelet\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:28.992644 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992613    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/da7b0d9d-cbe8-4f97-87e0-806003cd917b-socket-dir\") pod \"aws-ebs-csi-driver-node-5wqmk\" (UID: \"da7b0d9d-cbe8-4f97-87e0-806003cd917b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk"
Sep 19 01:12:28.993019 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992641    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5sr4\" (UniqueName: \"kubernetes.io/projected/54557d74-f7ed-44ac-b483-9942019a9c4d-kube-api-access-k5sr4\") pod \"node-resolver-grkk4\" (UID: \"54557d74-f7ed-44ac-b483-9942019a9c4d\") " pod="openshift-dns/node-resolver-grkk4"
Sep 19 01:12:28.993019 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992757    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3347022a-f11e-4bad-a7dd-902cf5613aa5-cni-binary-copy\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:28.993019 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992791    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhc2b\" (UniqueName: \"kubernetes.io/projected/da7b0d9d-cbe8-4f97-87e0-806003cd917b-kube-api-access-mhc2b\") pod \"aws-ebs-csi-driver-node-5wqmk\" (UID: \"da7b0d9d-cbe8-4f97-87e0-806003cd917b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk"
Sep 19 01:12:28.993019 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992816    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54557d74-f7ed-44ac-b483-9942019a9c4d-tmp-dir\") pod \"node-resolver-grkk4\" (UID: \"54557d74-f7ed-44ac-b483-9942019a9c4d\") " pod="openshift-dns/node-resolver-grkk4"
Sep 19 01:12:28.993019 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992840    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-hostroot\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:28.993019 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992864    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/da7b0d9d-cbe8-4f97-87e0-806003cd917b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5wqmk\" (UID: \"da7b0d9d-cbe8-4f97-87e0-806003cd917b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk"
Sep 19 01:12:28.993019 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992888    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/da7b0d9d-cbe8-4f97-87e0-806003cd917b-registration-dir\") pod \"aws-ebs-csi-driver-node-5wqmk\" (UID: \"da7b0d9d-cbe8-4f97-87e0-806003cd917b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk"
Sep 19 01:12:28.993019 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992913    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/56a6eadc-607c-4b80-8dc8-183e63551d8b-host-slash\") pod \"iptables-alerter-j6q6r\" (UID: \"56a6eadc-607c-4b80-8dc8-183e63551d8b\") " pod="openshift-network-operator/iptables-alerter-j6q6r"
Sep 19 01:12:28.993019 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992933    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-host-run-netns\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:28.993019 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992976    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-multus-conf-dir\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:28.993019 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.992998    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-etc-kubernetes\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:28.993520 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.993038    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 19 01:12:28.993520 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.993039    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-multus-cni-dir\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:28.993520 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.993112    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-host-run-k8s-cni-cncf-io\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:28.993520 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.993361    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 19 01:12:28.993710 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.993590    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 19 01:12:28.993710 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.993590    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 19 01:12:28.993910 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.993890    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 19 01:12:28.994184 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.994164    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-smspl\""
Sep 19 01:12:28.994382 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.994367    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 19 01:12:28.994461 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.994444    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dvd6m"
Sep 19 01:12:28.996507 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.996474    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 19 01:12:28.996705 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.996688    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5zjgs\""
Sep 19 01:12:28.996971 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.996919    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 19 01:12:28.997225 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.997196    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 19 01:12:28.997651 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.997631    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:12:28.997735 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:28.997706    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh6vq" podUID="538e796f-8768-4bf9-ab1c-c3631bb14317"
Sep 19 01:12:29.000078 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:28.999810    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.002449 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.002179    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7l5hk\""
Sep 19 01:12:29.010342 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.009765    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 19 01:12:29.010342 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.010064    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tfq72"
Sep 19 01:12:29.010342 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.010076    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 19 01:12:29.012318 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.012194    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9pxzt\""
Sep 19 01:12:29.013270 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.012938    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 19 01:12:29.013270 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.013137    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 19 01:12:29.013270 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.013197    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:29.013270 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:29.013260    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj96s" podUID="2cd09385-9d78-428c-bb46-efeb37e8e55a"
Sep 19 01:12:29.015102 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.015068    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:29.015191 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:29.015128    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hpjz8" podUID="01b79542-9c56-4f26-a1d4-c4f2257ff152"
Sep 19 01:12:29.020351 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.020331    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-n2whm"
Sep 19 01:12:29.028851 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.028831    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-n2whm"
Sep 19 01:12:29.064469 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.064357    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-76.ec2.internal" event={"ID":"60e7758b2d5dde524dbb14f374f071d5","Type":"ContainerStarted","Data":"e886711f166a6ff5f34ee74ab9cd19eb81abf55017939922404126fecdb26638"}
Sep 19 01:12:29.067104 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.067060    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-76.ec2.internal" event={"ID":"0034bf5044f034aee1efd2c2a680ecba","Type":"ContainerStarted","Data":"521891dc7a46fcd739f2187f31a1e174b86508c8089a46346f05914ed40332ea"}
Sep 19 01:12:29.082784 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.082765    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 19 01:12:29.093484 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.093457    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gwd9\" (UniqueName: \"kubernetes.io/projected/01b79542-9c56-4f26-a1d4-c4f2257ff152-kube-api-access-7gwd9\") pod \"network-metrics-daemon-hpjz8\" (UID: \"01b79542-9c56-4f26-a1d4-c4f2257ff152\") " pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:29.093595 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.093522    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/da7b0d9d-cbe8-4f97-87e0-806003cd917b-etc-selinux\") pod \"aws-ebs-csi-driver-node-5wqmk\" (UID: \"da7b0d9d-cbe8-4f97-87e0-806003cd917b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk"
Sep 19 01:12:29.093595 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.093550    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/da7b0d9d-cbe8-4f97-87e0-806003cd917b-sys-fs\") pod \"aws-ebs-csi-driver-node-5wqmk\" (UID: \"da7b0d9d-cbe8-4f97-87e0-806003cd917b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk"
Sep 19 01:12:29.093595 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.093575    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcbl4\" (UniqueName: \"kubernetes.io/projected/56a6eadc-607c-4b80-8dc8-183e63551d8b-kube-api-access-zcbl4\") pod \"iptables-alerter-j6q6r\" (UID: \"56a6eadc-607c-4b80-8dc8-183e63551d8b\") " pod="openshift-network-operator/iptables-alerter-j6q6r"
Sep 19 01:12:29.093797 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.093600    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-host-run-netns\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.093797 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.093623    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-etc-kubernetes\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.093797 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.093646    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-etc-sysctl-conf\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.093797 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.093671    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2f997c70-10b2-49e0-8711-00e914baa6c4-agent-certs\") pod \"konnectivity-agent-9n69k\" (UID: \"2f997c70-10b2-49e0-8711-00e914baa6c4\") " pod="kube-system/konnectivity-agent-9n69k"
Sep 19 01:12:29.093797 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.093694    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/54557d74-f7ed-44ac-b483-9942019a9c4d-hosts-file\") pod \"node-resolver-grkk4\" (UID: \"54557d74-f7ed-44ac-b483-9942019a9c4d\") " pod="openshift-dns/node-resolver-grkk4"
Sep 19 01:12:29.093797 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.093720    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/56a6eadc-607c-4b80-8dc8-183e63551d8b-iptables-alerter-script\") pod \"iptables-alerter-j6q6r\" (UID: \"56a6eadc-607c-4b80-8dc8-183e63551d8b\") " pod="openshift-network-operator/iptables-alerter-j6q6r"
Sep 19 01:12:29.093797 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.093744    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-run-openvswitch\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.093797 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.093766    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-log-socket\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.093797 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.093789    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbs7g\" (UniqueName: \"kubernetes.io/projected/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-kube-api-access-qbs7g\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.094201 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.093821    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-etc-sysctl-d\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.094201 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.093845    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-cnibin\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.094201 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.093871    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-host-var-lib-cni-multus\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.094201 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.093893    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-run\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.094201 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.093919    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2cd09385-9d78-428c-bb46-efeb37e8e55a-dbus\") pod \"global-pull-secret-syncer-kj96s\" (UID: \"2cd09385-9d78-428c-bb46-efeb37e8e55a\") " pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:29.094201 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.093944    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-host-var-lib-kubelet\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.094201 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.093998    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2857e44c-032d-4586-b438-26ad712aef9c-host\") pod \"node-ca-dvd6m\" (UID: \"2857e44c-032d-4586-b438-26ad712aef9c\") " pod="openshift-image-registry/node-ca-dvd6m"
Sep 19 01:12:29.094201 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094023    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/23fcc30f-348f-4c04-aff2-23fe3420219a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tfq72\" (UID: \"23fcc30f-348f-4c04-aff2-23fe3420219a\") " pod="openshift-multus/multus-additional-cni-plugins-tfq72"
Sep 19 01:12:29.094201 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094049    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01b79542-9c56-4f26-a1d4-c4f2257ff152-metrics-certs\") pod \"network-metrics-daemon-hpjz8\" (UID: \"01b79542-9c56-4f26-a1d4-c4f2257ff152\") " pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:29.094201 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094082    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-host-run-ovn-kubernetes\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.094201 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094105    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-lib-modules\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.094201 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094129    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/da7b0d9d-cbe8-4f97-87e0-806003cd917b-registration-dir\") pod \"aws-ebs-csi-driver-node-5wqmk\" (UID: \"da7b0d9d-cbe8-4f97-87e0-806003cd917b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk"
Sep 19 01:12:29.094201 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094158    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/56a6eadc-607c-4b80-8dc8-183e63551d8b-host-slash\") pod \"iptables-alerter-j6q6r\" (UID: \"56a6eadc-607c-4b80-8dc8-183e63551d8b\") " pod="openshift-network-operator/iptables-alerter-j6q6r"
Sep 19 01:12:29.094201 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094181    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-host-slash\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.095035 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094306    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-etc-modprobe-d\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.095035 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094340    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6894680-df25-4249-b9ef-c79e4bebdd3e-tmp\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.095035 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094363    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-env-overrides\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.095035 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094387    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-host-run-netns\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.095035 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094410    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-multus-conf-dir\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.095035 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094440    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-etc-openvswitch\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.095035 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094464    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-run-ovn\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.095035 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094476    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-host-var-lib-cni-multus\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.095035 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094487    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-host\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.095035 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094538    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-host-run-k8s-cni-cncf-io\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.095035 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094561    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrkx4\" (UniqueName: \"kubernetes.io/projected/3347022a-f11e-4bad-a7dd-902cf5613aa5-kube-api-access-jrkx4\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.095035 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094584    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thznt\" (UniqueName: \"kubernetes.io/projected/2857e44c-032d-4586-b438-26ad712aef9c-kube-api-access-thznt\") pod \"node-ca-dvd6m\" (UID: \"2857e44c-032d-4586-b438-26ad712aef9c\") " pod="openshift-image-registry/node-ca-dvd6m"
Sep 19 01:12:29.095035 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094588    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/da7b0d9d-cbe8-4f97-87e0-806003cd917b-registration-dir\") pod \"aws-ebs-csi-driver-node-5wqmk\" (UID: \"da7b0d9d-cbe8-4f97-87e0-806003cd917b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk"
Sep 19 01:12:29.095035 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094609    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23fcc30f-348f-4c04-aff2-23fe3420219a-system-cni-dir\") pod \"multus-additional-cni-plugins-tfq72\" (UID: \"23fcc30f-348f-4c04-aff2-23fe3420219a\") " pod="openshift-multus/multus-additional-cni-plugins-tfq72"
Sep 19 01:12:29.095035 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094624    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-cnibin\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.095035 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094639    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2cd09385-9d78-428c-bb46-efeb37e8e55a-kubelet-config\") pod \"global-pull-secret-syncer-kj96s\" (UID: \"2cd09385-9d78-428c-bb46-efeb37e8e55a\") " pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:29.095035 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094688    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2cd09385-9d78-428c-bb46-efeb37e8e55a-original-pull-secret\") pod \"global-pull-secret-syncer-kj96s\" (UID: \"2cd09385-9d78-428c-bb46-efeb37e8e55a\") " pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:29.095849 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094729    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-host-run-multus-certs\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.095849 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094751    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-ovnkube-script-lib\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.095849 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094774    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2857e44c-032d-4586-b438-26ad712aef9c-serviceca\") pod \"node-ca-dvd6m\" (UID: \"2857e44c-032d-4586-b438-26ad712aef9c\") " pod="openshift-image-registry/node-ca-dvd6m"
Sep 19 01:12:29.095849 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094775    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 19 01:12:29.095849 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094800    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-host-var-lib-cni-bin\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.095849 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094824    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3347022a-f11e-4bad-a7dd-902cf5613aa5-multus-daemon-config\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.095849 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094849    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/da7b0d9d-cbe8-4f97-87e0-806003cd917b-device-dir\") pod \"aws-ebs-csi-driver-node-5wqmk\" (UID: \"da7b0d9d-cbe8-4f97-87e0-806003cd917b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk"
Sep 19 01:12:29.095849 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094872    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-etc-systemd\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.095849 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094897    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-var-lib-kubelet\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.095849 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094910    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/da7b0d9d-cbe8-4f97-87e0-806003cd917b-sys-fs\") pod \"aws-ebs-csi-driver-node-5wqmk\" (UID: \"da7b0d9d-cbe8-4f97-87e0-806003cd917b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk"
Sep 19 01:12:29.095849 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094921    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2f997c70-10b2-49e0-8711-00e914baa6c4-konnectivity-ca\") pod \"konnectivity-agent-9n69k\" (UID: \"2f997c70-10b2-49e0-8711-00e914baa6c4\") " pod="kube-system/konnectivity-agent-9n69k"
Sep 19 01:12:29.095849 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094949    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-host-var-lib-kubelet\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.095849 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.094991    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-system-cni-dir\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.095849 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095005    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/56a6eadc-607c-4b80-8dc8-183e63551d8b-host-slash\") pod \"iptables-alerter-j6q6r\" (UID: \"56a6eadc-607c-4b80-8dc8-183e63551d8b\") " pod="openshift-network-operator/iptables-alerter-j6q6r"
Sep 19 01:12:29.095849 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095017    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-multus-socket-dir-parent\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.095849 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095047    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-run-systemd\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.095849 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095120    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/54557d74-f7ed-44ac-b483-9942019a9c4d-hosts-file\") pod \"node-resolver-grkk4\" (UID: \"54557d74-f7ed-44ac-b483-9942019a9c4d\") " pod="openshift-dns/node-resolver-grkk4"
Sep 19 01:12:29.095849 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095159    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/da7b0d9d-cbe8-4f97-87e0-806003cd917b-etc-selinux\") pod \"aws-ebs-csi-driver-node-5wqmk\" (UID: \"da7b0d9d-cbe8-4f97-87e0-806003cd917b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk"
Sep 19 01:12:29.096721 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095210    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/56a6eadc-607c-4b80-8dc8-183e63551d8b-iptables-alerter-script\") pod \"iptables-alerter-j6q6r\" (UID: \"56a6eadc-607c-4b80-8dc8-183e63551d8b\") " pod="openshift-network-operator/iptables-alerter-j6q6r"
Sep 19 01:12:29.096721 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095235    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-multus-socket-dir-parent\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.096721 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095278    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.096721 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095289    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-host-run-multus-certs\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.096721 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095306    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f6894680-df25-4249-b9ef-c79e4bebdd3e-etc-tuned\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.096721 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095331    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77q5s\" (UniqueName: \"kubernetes.io/projected/538e796f-8768-4bf9-ab1c-c3631bb14317-kube-api-access-77q5s\") pod \"network-check-target-nh6vq\" (UID: \"538e796f-8768-4bf9-ab1c-c3631bb14317\") " pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:12:29.096721 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095348    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-system-cni-dir\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.096721 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095361    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/da7b0d9d-cbe8-4f97-87e0-806003cd917b-socket-dir\") pod \"aws-ebs-csi-driver-node-5wqmk\" (UID: \"da7b0d9d-cbe8-4f97-87e0-806003cd917b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk"
Sep 19 01:12:29.096721 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5sr4\" (UniqueName: \"kubernetes.io/projected/54557d74-f7ed-44ac-b483-9942019a9c4d-kube-api-access-k5sr4\") pod \"node-resolver-grkk4\" (UID: \"54557d74-f7ed-44ac-b483-9942019a9c4d\") " pod="openshift-dns/node-resolver-grkk4"
Sep 19 01:12:29.096721 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095396    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-multus-conf-dir\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.096721 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095414    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-host-kubelet\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.096721 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095438    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23fcc30f-348f-4c04-aff2-23fe3420219a-cni-binary-copy\") pod \"multus-additional-cni-plugins-tfq72\" (UID: \"23fcc30f-348f-4c04-aff2-23fe3420219a\") " pod="openshift-multus/multus-additional-cni-plugins-tfq72"
Sep 19 01:12:29.096721 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095447    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-host-var-lib-cni-bin\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.096721 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095463    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23fcc30f-348f-4c04-aff2-23fe3420219a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tfq72\" (UID: \"23fcc30f-348f-4c04-aff2-23fe3420219a\") " pod="openshift-multus/multus-additional-cni-plugins-tfq72"
Sep 19 01:12:29.096721 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095482    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-host-run-netns\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.096721 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095490    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3347022a-f11e-4bad-a7dd-902cf5613aa5-cni-binary-copy\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.096721 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095536    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhc2b\" (UniqueName: \"kubernetes.io/projected/da7b0d9d-cbe8-4f97-87e0-806003cd917b-kube-api-access-mhc2b\") pod \"aws-ebs-csi-driver-node-5wqmk\" (UID: \"da7b0d9d-cbe8-4f97-87e0-806003cd917b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk"
Sep 19 01:12:29.097440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095561    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54557d74-f7ed-44ac-b483-9942019a9c4d-tmp-dir\") pod \"node-resolver-grkk4\" (UID: \"54557d74-f7ed-44ac-b483-9942019a9c4d\") " pod="openshift-dns/node-resolver-grkk4"
Sep 19 01:12:29.097440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095584    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-systemd-units\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.097440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095674    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-host-cni-netd\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.097440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095702    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-ovn-node-metrics-cert\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.097440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095726    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-etc-sysconfig\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.097440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095750    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2f997c70-10b2-49e0-8711-00e914baa6c4-konnectivity-ca\") pod \"konnectivity-agent-9n69k\" (UID: \"2f997c70-10b2-49e0-8711-00e914baa6c4\") " pod="kube-system/konnectivity-agent-9n69k"
Sep 19 01:12:29.097440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095750    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/23fcc30f-348f-4c04-aff2-23fe3420219a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tfq72\" (UID: \"23fcc30f-348f-4c04-aff2-23fe3420219a\") " pod="openshift-multus/multus-additional-cni-plugins-tfq72"
Sep 19 01:12:29.097440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095797    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/da7b0d9d-cbe8-4f97-87e0-806003cd917b-device-dir\") pod \"aws-ebs-csi-driver-node-5wqmk\" (UID: \"da7b0d9d-cbe8-4f97-87e0-806003cd917b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk"
Sep 19 01:12:29.097440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095815    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-hostroot\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.097440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095837    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-host-run-k8s-cni-cncf-io\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.097440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095843    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/da7b0d9d-cbe8-4f97-87e0-806003cd917b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5wqmk\" (UID: \"da7b0d9d-cbe8-4f97-87e0-806003cd917b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk"
Sep 19 01:12:29.097440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095872    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-sys\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.097440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095897    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-etc-kubernetes\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.097440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095921    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-var-lib-openvswitch\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.097440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095945    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-node-log\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.097440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095969    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-ovnkube-config\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.097440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.095987    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3347022a-f11e-4bad-a7dd-902cf5613aa5-multus-daemon-config\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.098179 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.096042    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ktvs\" (UniqueName: \"kubernetes.io/projected/f6894680-df25-4249-b9ef-c79e4bebdd3e-kube-api-access-4ktvs\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.098179 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.096071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-multus-cni-dir\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.098179 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.096092    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/da7b0d9d-cbe8-4f97-87e0-806003cd917b-socket-dir\") pod \"aws-ebs-csi-driver-node-5wqmk\" (UID: \"da7b0d9d-cbe8-4f97-87e0-806003cd917b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk"
Sep 19 01:12:29.098179 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.096096    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-host-cni-bin\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.098179 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.096142    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-hostroot\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.098179 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.096145    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23fcc30f-348f-4c04-aff2-23fe3420219a-cnibin\") pod \"multus-additional-cni-plugins-tfq72\" (UID: \"23fcc30f-348f-4c04-aff2-23fe3420219a\") " pod="openshift-multus/multus-additional-cni-plugins-tfq72"
Sep 19 01:12:29.098179 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.096174    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f87w\" (UniqueName: \"kubernetes.io/projected/23fcc30f-348f-4c04-aff2-23fe3420219a-kube-api-access-2f87w\") pod \"multus-additional-cni-plugins-tfq72\" (UID: \"23fcc30f-348f-4c04-aff2-23fe3420219a\") " pod="openshift-multus/multus-additional-cni-plugins-tfq72"
Sep 19 01:12:29.098179 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.096184    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/da7b0d9d-cbe8-4f97-87e0-806003cd917b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5wqmk\" (UID: \"da7b0d9d-cbe8-4f97-87e0-806003cd917b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk"
Sep 19 01:12:29.098179 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.096221    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-etc-kubernetes\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.098179 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.096355    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3347022a-f11e-4bad-a7dd-902cf5613aa5-multus-cni-dir\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.098179 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.096441    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3347022a-f11e-4bad-a7dd-902cf5613aa5-cni-binary-copy\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.098179 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.096686    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54557d74-f7ed-44ac-b483-9942019a9c4d-tmp-dir\") pod \"node-resolver-grkk4\" (UID: \"54557d74-f7ed-44ac-b483-9942019a9c4d\") " pod="openshift-dns/node-resolver-grkk4"
Sep 19 01:12:29.100697 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.100353    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2f997c70-10b2-49e0-8711-00e914baa6c4-agent-certs\") pod \"konnectivity-agent-9n69k\" (UID: \"2f997c70-10b2-49e0-8711-00e914baa6c4\") " pod="kube-system/konnectivity-agent-9n69k"
Sep 19 01:12:29.109422 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.109361    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhc2b\" (UniqueName: \"kubernetes.io/projected/da7b0d9d-cbe8-4f97-87e0-806003cd917b-kube-api-access-mhc2b\") pod \"aws-ebs-csi-driver-node-5wqmk\" (UID: \"da7b0d9d-cbe8-4f97-87e0-806003cd917b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk"
Sep 19 01:12:29.111852 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.111815    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5sr4\" (UniqueName: \"kubernetes.io/projected/54557d74-f7ed-44ac-b483-9942019a9c4d-kube-api-access-k5sr4\") pod \"node-resolver-grkk4\" (UID: \"54557d74-f7ed-44ac-b483-9942019a9c4d\") " pod="openshift-dns/node-resolver-grkk4"
Sep 19 01:12:29.115245 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.115204    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcbl4\" (UniqueName: \"kubernetes.io/projected/56a6eadc-607c-4b80-8dc8-183e63551d8b-kube-api-access-zcbl4\") pod \"iptables-alerter-j6q6r\" (UID: \"56a6eadc-607c-4b80-8dc8-183e63551d8b\") " pod="openshift-network-operator/iptables-alerter-j6q6r"
Sep 19 01:12:29.116771 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.116751    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrkx4\" (UniqueName: \"kubernetes.io/projected/3347022a-f11e-4bad-a7dd-902cf5613aa5-kube-api-access-jrkx4\") pod \"multus-d4nk2\" (UID: \"3347022a-f11e-4bad-a7dd-902cf5613aa5\") " pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.196707 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.196679    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-systemd-units\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.196707 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.196717    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-host-cni-netd\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.196893 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.196743    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-ovn-node-metrics-cert\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.196893 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.196750    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-systemd-units\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.196893 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.196768    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-etc-sysconfig\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.196893 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.196792    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/23fcc30f-348f-4c04-aff2-23fe3420219a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tfq72\" (UID: \"23fcc30f-348f-4c04-aff2-23fe3420219a\") " pod="openshift-multus/multus-additional-cni-plugins-tfq72"
Sep 19 01:12:29.196893 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.196818    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-sys\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.196893 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.196824    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-host-cni-netd\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.196893 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.196841    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-var-lib-openvswitch\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.196893 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.196864    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-node-log\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.196893 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.196885    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-ovnkube-config\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.197295 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.196906    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ktvs\" (UniqueName: \"kubernetes.io/projected/f6894680-df25-4249-b9ef-c79e4bebdd3e-kube-api-access-4ktvs\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.197295 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.196929    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-host-cni-bin\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.197295 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.196950    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23fcc30f-348f-4c04-aff2-23fe3420219a-cnibin\") pod \"multus-additional-cni-plugins-tfq72\" (UID: \"23fcc30f-348f-4c04-aff2-23fe3420219a\") " pod="openshift-multus/multus-additional-cni-plugins-tfq72"
Sep 19 01:12:29.197295 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.196971    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f87w\" (UniqueName: \"kubernetes.io/projected/23fcc30f-348f-4c04-aff2-23fe3420219a-kube-api-access-2f87w\") pod \"multus-additional-cni-plugins-tfq72\" (UID: \"23fcc30f-348f-4c04-aff2-23fe3420219a\") " pod="openshift-multus/multus-additional-cni-plugins-tfq72"
Sep 19 01:12:29.197295 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.196994    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gwd9\" (UniqueName: \"kubernetes.io/projected/01b79542-9c56-4f26-a1d4-c4f2257ff152-kube-api-access-7gwd9\") pod \"network-metrics-daemon-hpjz8\" (UID: \"01b79542-9c56-4f26-a1d4-c4f2257ff152\") " pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:29.197295 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197020    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-host-run-netns\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.197295 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197044    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-etc-kubernetes\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.197295 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197067    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-etc-sysctl-conf\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.197295 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-run-openvswitch\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.197295 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-log-socket\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.197295 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197139    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbs7g\" (UniqueName: \"kubernetes.io/projected/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-kube-api-access-qbs7g\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.197295 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197160    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-etc-sysctl-d\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.197295 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197184    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-run\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.197295 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197207    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2cd09385-9d78-428c-bb46-efeb37e8e55a-dbus\") pod \"global-pull-secret-syncer-kj96s\" (UID: \"2cd09385-9d78-428c-bb46-efeb37e8e55a\") " pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:29.197295 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2857e44c-032d-4586-b438-26ad712aef9c-host\") pod \"node-ca-dvd6m\" (UID: \"2857e44c-032d-4586-b438-26ad712aef9c\") " pod="openshift-image-registry/node-ca-dvd6m"
Sep 19 01:12:29.197295 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197261    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/23fcc30f-348f-4c04-aff2-23fe3420219a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tfq72\" (UID: \"23fcc30f-348f-4c04-aff2-23fe3420219a\") " pod="openshift-multus/multus-additional-cni-plugins-tfq72"
Sep 19 01:12:29.197295 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197286    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01b79542-9c56-4f26-a1d4-c4f2257ff152-metrics-certs\") pod \"network-metrics-daemon-hpjz8\" (UID: \"01b79542-9c56-4f26-a1d4-c4f2257ff152\") " pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:29.198157 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197311    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-host-run-ovn-kubernetes\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.198157 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197334    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-lib-modules\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.198157 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197359    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-host-slash\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.198157 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197365    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/23fcc30f-348f-4c04-aff2-23fe3420219a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tfq72\" (UID: \"23fcc30f-348f-4c04-aff2-23fe3420219a\") " pod="openshift-multus/multus-additional-cni-plugins-tfq72"
Sep 19 01:12:29.198157 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197382    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-etc-modprobe-d\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.198157 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197416    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6894680-df25-4249-b9ef-c79e4bebdd3e-tmp\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.198157 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197439    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-sys\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.198157 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197486    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-var-lib-openvswitch\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.198157 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197550    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-node-log\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.198157 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197443    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-env-overrides\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.198157 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197603    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-etc-openvswitch\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.198157 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197629    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-host-run-netns\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.198157 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197687    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-run-ovn\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.198157 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197691    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-run\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.198157 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:29.197695    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 01:12:29.198157 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197710    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-lib-modules\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.198157 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197749    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2cd09385-9d78-428c-bb46-efeb37e8e55a-dbus\") pod \"global-pull-secret-syncer-kj96s\" (UID: \"2cd09385-9d78-428c-bb46-efeb37e8e55a\") " pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:29.198157 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197754    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-host-run-ovn-kubernetes\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.199114 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:29.197769    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01b79542-9c56-4f26-a1d4-c4f2257ff152-metrics-certs podName:01b79542-9c56-4f26-a1d4-c4f2257ff152 nodeName:}" failed. No retries permitted until 2026-09-19 01:12:29.697748366 +0000 UTC m=+4.604610411 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/01b79542-9c56-4f26-a1d4-c4f2257ff152-metrics-certs") pod "network-metrics-daemon-hpjz8" (UID: "01b79542-9c56-4f26-a1d4-c4f2257ff152") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 01:12:29.199114 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197799    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-host-slash\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.199114 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197803    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-etc-openvswitch\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.199114 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197610    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2857e44c-032d-4586-b438-26ad712aef9c-host\") pod \"node-ca-dvd6m\" (UID: \"2857e44c-032d-4586-b438-26ad712aef9c\") " pod="openshift-image-registry/node-ca-dvd6m"
Sep 19 01:12:29.199114 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197814    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-etc-sysctl-conf\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.199114 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197834    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23fcc30f-348f-4c04-aff2-23fe3420219a-cnibin\") pod \"multus-additional-cni-plugins-tfq72\" (UID: \"23fcc30f-348f-4c04-aff2-23fe3420219a\") " pod="openshift-multus/multus-additional-cni-plugins-tfq72"
Sep 19 01:12:29.199114 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197844    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-run-openvswitch\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.199114 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197856    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-host-cni-bin\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.199114 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197884    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-log-socket\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.199114 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197951    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-etc-sysctl-d\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.199114 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197640    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-run-ovn\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.199114 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.197995    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-host\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.199114 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198023    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thznt\" (UniqueName: \"kubernetes.io/projected/2857e44c-032d-4586-b438-26ad712aef9c-kube-api-access-thznt\") pod \"node-ca-dvd6m\" (UID: \"2857e44c-032d-4586-b438-26ad712aef9c\") " pod="openshift-image-registry/node-ca-dvd6m"
Sep 19 01:12:29.199114 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198043    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/23fcc30f-348f-4c04-aff2-23fe3420219a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tfq72\" (UID: \"23fcc30f-348f-4c04-aff2-23fe3420219a\") " pod="openshift-multus/multus-additional-cni-plugins-tfq72"
Sep 19 01:12:29.199114 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198044    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-ovnkube-config\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.199114 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198047    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23fcc30f-348f-4c04-aff2-23fe3420219a-system-cni-dir\") pod \"multus-additional-cni-plugins-tfq72\" (UID: \"23fcc30f-348f-4c04-aff2-23fe3420219a\") " pod="openshift-multus/multus-additional-cni-plugins-tfq72"
Sep 19 01:12:29.199114 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198099    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2cd09385-9d78-428c-bb46-efeb37e8e55a-kubelet-config\") pod \"global-pull-secret-syncer-kj96s\" (UID: \"2cd09385-9d78-428c-bb46-efeb37e8e55a\") " pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:29.199917 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198107    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-etc-kubernetes\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.199917 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198120    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-etc-modprobe-d\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.199917 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198124    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2cd09385-9d78-428c-bb46-efeb37e8e55a-original-pull-secret\") pod \"global-pull-secret-syncer-kj96s\" (UID: \"2cd09385-9d78-428c-bb46-efeb37e8e55a\") " pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:29.199917 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198151    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-ovnkube-script-lib\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.199917 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198165    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2cd09385-9d78-428c-bb46-efeb37e8e55a-kubelet-config\") pod \"global-pull-secret-syncer-kj96s\" (UID: \"2cd09385-9d78-428c-bb46-efeb37e8e55a\") " pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:29.199917 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2857e44c-032d-4586-b438-26ad712aef9c-serviceca\") pod \"node-ca-dvd6m\" (UID: \"2857e44c-032d-4586-b438-26ad712aef9c\") " pod="openshift-image-registry/node-ca-dvd6m"
Sep 19 01:12:29.199917 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198203    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-etc-systemd\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.199917 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198209    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-host\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.199917 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198225    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-var-lib-kubelet\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.199917 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198248    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23fcc30f-348f-4c04-aff2-23fe3420219a-system-cni-dir\") pod \"multus-additional-cni-plugins-tfq72\" (UID: \"23fcc30f-348f-4c04-aff2-23fe3420219a\") " pod="openshift-multus/multus-additional-cni-plugins-tfq72"
Sep 19 01:12:29.199917 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198253    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-run-systemd\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.199917 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198293    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-run-systemd\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.199917 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:29.198323    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 01:12:29.199917 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:29.198376    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cd09385-9d78-428c-bb46-efeb37e8e55a-original-pull-secret podName:2cd09385-9d78-428c-bb46-efeb37e8e55a nodeName:}" failed. No retries permitted until 2026-09-19 01:12:29.69836102 +0000 UTC m=+4.605223065 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2cd09385-9d78-428c-bb46-efeb37e8e55a-original-pull-secret") pod "global-pull-secret-syncer-kj96s" (UID: "2cd09385-9d78-428c-bb46-efeb37e8e55a") : object "kube-system"/"original-pull-secret" not registered
Sep 19 01:12:29.199917 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198409    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.199917 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198421    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-env-overrides\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.199917 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198442    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f6894680-df25-4249-b9ef-c79e4bebdd3e-etc-tuned\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.200402 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198469    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77q5s\" (UniqueName: \"kubernetes.io/projected/538e796f-8768-4bf9-ab1c-c3631bb14317-kube-api-access-77q5s\") pod \"network-check-target-nh6vq\" (UID: \"538e796f-8768-4bf9-ab1c-c3631bb14317\") " pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:12:29.200402 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198473    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.200402 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198489    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-var-lib-kubelet\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.200402 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198442    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-etc-systemd\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.200402 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198516    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-host-kubelet\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.200402 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198560    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23fcc30f-348f-4c04-aff2-23fe3420219a-cni-binary-copy\") pod \"multus-additional-cni-plugins-tfq72\" (UID: \"23fcc30f-348f-4c04-aff2-23fe3420219a\") " pod="openshift-multus/multus-additional-cni-plugins-tfq72"
Sep 19 01:12:29.200402 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198586    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23fcc30f-348f-4c04-aff2-23fe3420219a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tfq72\" (UID: \"23fcc30f-348f-4c04-aff2-23fe3420219a\") " pod="openshift-multus/multus-additional-cni-plugins-tfq72"
Sep 19 01:12:29.200402 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198728    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2857e44c-032d-4586-b438-26ad712aef9c-serviceca\") pod \"node-ca-dvd6m\" (UID: \"2857e44c-032d-4586-b438-26ad712aef9c\") " pod="openshift-image-registry/node-ca-dvd6m"
Sep 19 01:12:29.200402 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198747    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23fcc30f-348f-4c04-aff2-23fe3420219a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tfq72\" (UID: \"23fcc30f-348f-4c04-aff2-23fe3420219a\") " pod="openshift-multus/multus-additional-cni-plugins-tfq72"
Sep 19 01:12:29.200402 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198768    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-ovnkube-script-lib\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.200402 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198779    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f6894680-df25-4249-b9ef-c79e4bebdd3e-etc-sysconfig\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.200402 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.198902    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-host-kubelet\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.200402 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.199134    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23fcc30f-348f-4c04-aff2-23fe3420219a-cni-binary-copy\") pod \"multus-additional-cni-plugins-tfq72\" (UID: \"23fcc30f-348f-4c04-aff2-23fe3420219a\") " pod="openshift-multus/multus-additional-cni-plugins-tfq72"
Sep 19 01:12:29.200402 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.200152    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-ovn-node-metrics-cert\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.200402 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.200355    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6894680-df25-4249-b9ef-c79e4bebdd3e-tmp\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.200885 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.200696    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f6894680-df25-4249-b9ef-c79e4bebdd3e-etc-tuned\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.208641 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.208613    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ktvs\" (UniqueName: \"kubernetes.io/projected/f6894680-df25-4249-b9ef-c79e4bebdd3e-kube-api-access-4ktvs\") pod \"tuned-46sks\" (UID: \"f6894680-df25-4249-b9ef-c79e4bebdd3e\") " pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.208948 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.208925    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f87w\" (UniqueName: \"kubernetes.io/projected/23fcc30f-348f-4c04-aff2-23fe3420219a-kube-api-access-2f87w\") pod \"multus-additional-cni-plugins-tfq72\" (UID: \"23fcc30f-348f-4c04-aff2-23fe3420219a\") " pod="openshift-multus/multus-additional-cni-plugins-tfq72"
Sep 19 01:12:29.217067 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.217046    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thznt\" (UniqueName: \"kubernetes.io/projected/2857e44c-032d-4586-b438-26ad712aef9c-kube-api-access-thznt\") pod \"node-ca-dvd6m\" (UID: \"2857e44c-032d-4586-b438-26ad712aef9c\") " pod="openshift-image-registry/node-ca-dvd6m"
Sep 19 01:12:29.217155 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:29.217102    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 01:12:29.217155 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:29.217117    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 01:12:29.217155 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:29.217126    2580 projected.go:196] Error preparing data for projected volume kube-api-access-77q5s for pod openshift-network-diagnostics/network-check-target-nh6vq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 01:12:29.217309 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:29.217184    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/538e796f-8768-4bf9-ab1c-c3631bb14317-kube-api-access-77q5s podName:538e796f-8768-4bf9-ab1c-c3631bb14317 nodeName:}" failed. No retries permitted until 2026-09-19 01:12:29.71717096 +0000 UTC m=+4.624032990 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-77q5s" (UniqueName: "kubernetes.io/projected/538e796f-8768-4bf9-ab1c-c3631bb14317-kube-api-access-77q5s") pod "network-check-target-nh6vq" (UID: "538e796f-8768-4bf9-ab1c-c3631bb14317") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 01:12:29.219300 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.219282    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbs7g\" (UniqueName: \"kubernetes.io/projected/888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4-kube-api-access-qbs7g\") pod \"ovnkube-node-7z77w\" (UID: \"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.219394 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.219348    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gwd9\" (UniqueName: \"kubernetes.io/projected/01b79542-9c56-4f26-a1d4-c4f2257ff152-kube-api-access-7gwd9\") pod \"network-metrics-daemon-hpjz8\" (UID: \"01b79542-9c56-4f26-a1d4-c4f2257ff152\") " pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:29.293781 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.293751    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9n69k"
Sep 19 01:12:29.311055 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.311030    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d4nk2"
Sep 19 01:12:29.324992 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.324828    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk"
Sep 19 01:12:29.329552 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.329531    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-grkk4"
Sep 19 01:12:29.338133 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.338112    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j6q6r"
Sep 19 01:12:29.346844 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.346826    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:29.354527 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.354461    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dvd6m"
Sep 19 01:12:29.362008 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.361991    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-46sks"
Sep 19 01:12:29.369660 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.369641    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tfq72"
Sep 19 01:12:29.701318 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.701233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01b79542-9c56-4f26-a1d4-c4f2257ff152-metrics-certs\") pod \"network-metrics-daemon-hpjz8\" (UID: \"01b79542-9c56-4f26-a1d4-c4f2257ff152\") " pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:29.701318 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.701284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2cd09385-9d78-428c-bb46-efeb37e8e55a-original-pull-secret\") pod \"global-pull-secret-syncer-kj96s\" (UID: \"2cd09385-9d78-428c-bb46-efeb37e8e55a\") " pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:29.701536 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:29.701384    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 01:12:29.701536 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:29.701386    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 01:12:29.701536 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:29.701439    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cd09385-9d78-428c-bb46-efeb37e8e55a-original-pull-secret podName:2cd09385-9d78-428c-bb46-efeb37e8e55a nodeName:}" failed. No retries permitted until 2026-09-19 01:12:30.7014258 +0000 UTC m=+5.608287825 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2cd09385-9d78-428c-bb46-efeb37e8e55a-original-pull-secret") pod "global-pull-secret-syncer-kj96s" (UID: "2cd09385-9d78-428c-bb46-efeb37e8e55a") : object "kube-system"/"original-pull-secret" not registered
Sep 19 01:12:29.701536 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:29.701454    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01b79542-9c56-4f26-a1d4-c4f2257ff152-metrics-certs podName:01b79542-9c56-4f26-a1d4-c4f2257ff152 nodeName:}" failed. No retries permitted until 2026-09-19 01:12:30.701447944 +0000 UTC m=+5.608309970 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/01b79542-9c56-4f26-a1d4-c4f2257ff152-metrics-certs") pod "network-metrics-daemon-hpjz8" (UID: "01b79542-9c56-4f26-a1d4-c4f2257ff152") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 01:12:29.802061 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:29.802019    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77q5s\" (UniqueName: \"kubernetes.io/projected/538e796f-8768-4bf9-ab1c-c3631bb14317-kube-api-access-77q5s\") pod \"network-check-target-nh6vq\" (UID: \"538e796f-8768-4bf9-ab1c-c3631bb14317\") " pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:12:29.802227 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:29.802203    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 01:12:29.802227 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:29.802227    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 01:12:29.802334 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:29.802240    2580 projected.go:196] Error preparing data for projected volume kube-api-access-77q5s for pod openshift-network-diagnostics/network-check-target-nh6vq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 01:12:29.802334 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:29.802305    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/538e796f-8768-4bf9-ab1c-c3631bb14317-kube-api-access-77q5s podName:538e796f-8768-4bf9-ab1c-c3631bb14317 nodeName:}" failed. No retries permitted until 2026-09-19 01:12:30.802285627 +0000 UTC m=+5.709147673 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-77q5s" (UniqueName: "kubernetes.io/projected/538e796f-8768-4bf9-ab1c-c3631bb14317-kube-api-access-77q5s") pod "network-check-target-nh6vq" (UID: "538e796f-8768-4bf9-ab1c-c3631bb14317") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 01:12:30.031967 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:30.031929    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 01:07:29 +0000 UTC" deadline="2028-03-12 04:04:01.499774826 +0000 UTC"
Sep 19 01:12:30.031967 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:30.031962    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12962h51m31.467816838s"
Sep 19 01:12:30.238734 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:30.238704    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23fcc30f_348f_4c04_aff2_23fe3420219a.slice/crio-df33e1b309e96ed82a092a056c03b215b3fe32fe08ef202f4ca17ef1819f9351 WatchSource:0}: Error finding container df33e1b309e96ed82a092a056c03b215b3fe32fe08ef202f4ca17ef1819f9351: Status 404 returned error can't find the container with id df33e1b309e96ed82a092a056c03b215b3fe32fe08ef202f4ca17ef1819f9351
Sep 19 01:12:30.240021 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:30.239981    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54557d74_f7ed_44ac_b483_9942019a9c4d.slice/crio-aa2cc32ebb943010e8608c9f4a3d4395de41a5ea265f61b852a9490625e56f08 WatchSource:0}: Error finding container aa2cc32ebb943010e8608c9f4a3d4395de41a5ea265f61b852a9490625e56f08: Status 404 returned error can't find the container with id aa2cc32ebb943010e8608c9f4a3d4395de41a5ea265f61b852a9490625e56f08
Sep 19 01:12:30.243546 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:30.243525    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod888ee4fc_23e7_4ee4_ac81_dd9f21a09fb4.slice/crio-68de3414d879d247d1477c1c1a60eae514eff24d0afea5679301d9c67e188646 WatchSource:0}: Error finding container 68de3414d879d247d1477c1c1a60eae514eff24d0afea5679301d9c67e188646: Status 404 returned error can't find the container with id 68de3414d879d247d1477c1c1a60eae514eff24d0afea5679301d9c67e188646
Sep 19 01:12:30.244545 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:30.244481    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3347022a_f11e_4bad_a7dd_902cf5613aa5.slice/crio-735e311ccd8e726e7fdf54730d4468ee2d2bf35e0e2679f9d85914572e8f5147 WatchSource:0}: Error finding container 735e311ccd8e726e7fdf54730d4468ee2d2bf35e0e2679f9d85914572e8f5147: Status 404 returned error can't find the container with id 735e311ccd8e726e7fdf54730d4468ee2d2bf35e0e2679f9d85914572e8f5147
Sep 19 01:12:30.246113 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:30.246038    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f997c70_10b2_49e0_8711_00e914baa6c4.slice/crio-e9508dbb4a9943be11af594f36c909b035b121ec4927ab0559f54433a0daa697 WatchSource:0}: Error finding container e9508dbb4a9943be11af594f36c909b035b121ec4927ab0559f54433a0daa697: Status 404 returned error can't find the container with id e9508dbb4a9943be11af594f36c909b035b121ec4927ab0559f54433a0daa697
Sep 19 01:12:30.246680 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:30.246637    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56a6eadc_607c_4b80_8dc8_183e63551d8b.slice/crio-5220da1bb89c86c541e5ddacd04952b260ab8993b2f06488d35f8dad577ee045 WatchSource:0}: Error finding container 5220da1bb89c86c541e5ddacd04952b260ab8993b2f06488d35f8dad577ee045: Status 404 returned error can't find the container with id 5220da1bb89c86c541e5ddacd04952b260ab8993b2f06488d35f8dad577ee045
Sep 19 01:12:30.247982 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:30.247924    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6894680_df25_4249_b9ef_c79e4bebdd3e.slice/crio-00a102b091ed3c630454fc5f92092a4fc8556dd552a1cc4e8926845913a08f9d WatchSource:0}: Error finding container 00a102b091ed3c630454fc5f92092a4fc8556dd552a1cc4e8926845913a08f9d: Status 404 returned error can't find the container with id 00a102b091ed3c630454fc5f92092a4fc8556dd552a1cc4e8926845913a08f9d
Sep 19 01:12:30.248400 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:30.248381    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda7b0d9d_cbe8_4f97_87e0_806003cd917b.slice/crio-ca572806f4bbec6438490a0154c2ee45a007be9ad7204c106830514060640b07 WatchSource:0}: Error finding container ca572806f4bbec6438490a0154c2ee45a007be9ad7204c106830514060640b07: Status 404 returned error can't find the container with id ca572806f4bbec6438490a0154c2ee45a007be9ad7204c106830514060640b07
Sep 19 01:12:30.250367 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:12:30.250346    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2857e44c_032d_4586_b438_26ad712aef9c.slice/crio-f0f3e024fef2e07958fe2bc2952db7406b0c482ab66698c3b5851a5434811941 WatchSource:0}: Error finding container f0f3e024fef2e07958fe2bc2952db7406b0c482ab66698c3b5851a5434811941: Status 404 returned error can't find the container with id f0f3e024fef2e07958fe2bc2952db7406b0c482ab66698c3b5851a5434811941
Sep 19 01:12:30.708103 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:30.708013    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2cd09385-9d78-428c-bb46-efeb37e8e55a-original-pull-secret\") pod \"global-pull-secret-syncer-kj96s\" (UID: \"2cd09385-9d78-428c-bb46-efeb37e8e55a\") " pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:30.708265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:30.708113    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01b79542-9c56-4f26-a1d4-c4f2257ff152-metrics-certs\") pod \"network-metrics-daemon-hpjz8\" (UID: \"01b79542-9c56-4f26-a1d4-c4f2257ff152\") " pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:30.708265 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:30.708190    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 01:12:30.708265 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:30.708228    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 01:12:30.708415 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:30.708279    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cd09385-9d78-428c-bb46-efeb37e8e55a-original-pull-secret podName:2cd09385-9d78-428c-bb46-efeb37e8e55a nodeName:}" failed. No retries permitted until 2026-09-19 01:12:32.708258993 +0000 UTC m=+7.615121032 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2cd09385-9d78-428c-bb46-efeb37e8e55a-original-pull-secret") pod "global-pull-secret-syncer-kj96s" (UID: "2cd09385-9d78-428c-bb46-efeb37e8e55a") : object "kube-system"/"original-pull-secret" not registered
Sep 19 01:12:30.708415 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:30.708301    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01b79542-9c56-4f26-a1d4-c4f2257ff152-metrics-certs podName:01b79542-9c56-4f26-a1d4-c4f2257ff152 nodeName:}" failed. No retries permitted until 2026-09-19 01:12:32.708290617 +0000 UTC m=+7.615152644 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/01b79542-9c56-4f26-a1d4-c4f2257ff152-metrics-certs") pod "network-metrics-daemon-hpjz8" (UID: "01b79542-9c56-4f26-a1d4-c4f2257ff152") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 01:12:30.808928 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:30.808885    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77q5s\" (UniqueName: \"kubernetes.io/projected/538e796f-8768-4bf9-ab1c-c3631bb14317-kube-api-access-77q5s\") pod \"network-check-target-nh6vq\" (UID: \"538e796f-8768-4bf9-ab1c-c3631bb14317\") " pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:12:30.809084 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:30.809065    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 01:12:30.809137 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:30.809091    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 01:12:30.809137 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:30.809105    2580 projected.go:196] Error preparing data for projected volume kube-api-access-77q5s for pod openshift-network-diagnostics/network-check-target-nh6vq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 01:12:30.809247 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:30.809165    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/538e796f-8768-4bf9-ab1c-c3631bb14317-kube-api-access-77q5s podName:538e796f-8768-4bf9-ab1c-c3631bb14317 nodeName:}" failed. No retries permitted until 2026-09-19 01:12:32.80914711 +0000 UTC m=+7.716009150 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-77q5s" (UniqueName: "kubernetes.io/projected/538e796f-8768-4bf9-ab1c-c3631bb14317-kube-api-access-77q5s") pod "network-check-target-nh6vq" (UID: "538e796f-8768-4bf9-ab1c-c3631bb14317") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 01:12:31.033149 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:31.033089    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 01:07:29 +0000 UTC" deadline="2028-04-07 10:41:22.274722088 +0000 UTC"
Sep 19 01:12:31.033149 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:31.033147    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13593h28m51.241579293s"
Sep 19 01:12:31.057565 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:31.057538    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:31.057703 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:31.057665    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj96s" podUID="2cd09385-9d78-428c-bb46-efeb37e8e55a"
Sep 19 01:12:31.058154 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:31.058132    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:12:31.058261 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:31.058230    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh6vq" podUID="538e796f-8768-4bf9-ab1c-c3631bb14317"
Sep 19 01:12:31.058332 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:31.058313    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:31.058449 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:31.058430    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hpjz8" podUID="01b79542-9c56-4f26-a1d4-c4f2257ff152"
Sep 19 01:12:31.083928 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:31.083896    2580 generic.go:356] "Generic (PLEG): container finished" podID="60e7758b2d5dde524dbb14f374f071d5" containerID="6b91d88c390b0877fb3d0684e38bc96d91bb633e92bf75e12c281e5d7a4aa00a" exitCode=0
Sep 19 01:12:31.084102 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:31.084006    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-76.ec2.internal" event={"ID":"60e7758b2d5dde524dbb14f374f071d5","Type":"ContainerDied","Data":"6b91d88c390b0877fb3d0684e38bc96d91bb633e92bf75e12c281e5d7a4aa00a"}
Sep 19 01:12:31.095651 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:31.095299    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-76.ec2.internal" event={"ID":"0034bf5044f034aee1efd2c2a680ecba","Type":"ContainerStarted","Data":"ff9a2702c6d210f106045fb7867d9137252f3dd71267abb96a99a6e992be38c1"}
Sep 19 01:12:31.115683 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:31.115600    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk" event={"ID":"da7b0d9d-cbe8-4f97-87e0-806003cd917b","Type":"ContainerStarted","Data":"ca572806f4bbec6438490a0154c2ee45a007be9ad7204c106830514060640b07"}
Sep 19 01:12:31.118219 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:31.117770    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-76.ec2.internal" podStartSLOduration=3.11775374 podStartE2EDuration="3.11775374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 01:12:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:12:31.117562102 +0000 UTC m=+6.024424152" watchObservedRunningTime="2026-09-19 01:12:31.11775374 +0000 UTC m=+6.024615789"
Sep 19 01:12:31.121182 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:31.121107    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j6q6r" event={"ID":"56a6eadc-607c-4b80-8dc8-183e63551d8b","Type":"ContainerStarted","Data":"5220da1bb89c86c541e5ddacd04952b260ab8993b2f06488d35f8dad577ee045"}
Sep 19 01:12:31.127002 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:31.126941    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d4nk2" event={"ID":"3347022a-f11e-4bad-a7dd-902cf5613aa5","Type":"ContainerStarted","Data":"735e311ccd8e726e7fdf54730d4468ee2d2bf35e0e2679f9d85914572e8f5147"}
Sep 19 01:12:31.130581 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:31.130517    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-grkk4" event={"ID":"54557d74-f7ed-44ac-b483-9942019a9c4d","Type":"ContainerStarted","Data":"aa2cc32ebb943010e8608c9f4a3d4395de41a5ea265f61b852a9490625e56f08"}
Sep 19 01:12:31.134618 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:31.134594    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tfq72" event={"ID":"23fcc30f-348f-4c04-aff2-23fe3420219a","Type":"ContainerStarted","Data":"df33e1b309e96ed82a092a056c03b215b3fe32fe08ef202f4ca17ef1819f9351"}
Sep 19 01:12:31.147586 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:31.147527    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dvd6m" event={"ID":"2857e44c-032d-4586-b438-26ad712aef9c","Type":"ContainerStarted","Data":"f0f3e024fef2e07958fe2bc2952db7406b0c482ab66698c3b5851a5434811941"}
Sep 19 01:12:31.160152 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:31.160128    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-46sks" event={"ID":"f6894680-df25-4249-b9ef-c79e4bebdd3e","Type":"ContainerStarted","Data":"00a102b091ed3c630454fc5f92092a4fc8556dd552a1cc4e8926845913a08f9d"}
Sep 19 01:12:31.171357 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:31.171331    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9n69k" event={"ID":"2f997c70-10b2-49e0-8711-00e914baa6c4","Type":"ContainerStarted","Data":"e9508dbb4a9943be11af594f36c909b035b121ec4927ab0559f54433a0daa697"}
Sep 19 01:12:31.182985 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:31.182958    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7z77w" event={"ID":"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4","Type":"ContainerStarted","Data":"68de3414d879d247d1477c1c1a60eae514eff24d0afea5679301d9c67e188646"}
Sep 19 01:12:32.195637 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:32.195554    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-76.ec2.internal" event={"ID":"60e7758b2d5dde524dbb14f374f071d5","Type":"ContainerStarted","Data":"34489a32894bd58a98b484096ef8e1b785e6e583b792442b6c942f4a0a703b7c"}
Sep 19 01:12:32.725869 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:32.725119    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01b79542-9c56-4f26-a1d4-c4f2257ff152-metrics-certs\") pod \"network-metrics-daemon-hpjz8\" (UID: \"01b79542-9c56-4f26-a1d4-c4f2257ff152\") " pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:32.725869 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:32.725183    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2cd09385-9d78-428c-bb46-efeb37e8e55a-original-pull-secret\") pod \"global-pull-secret-syncer-kj96s\" (UID: \"2cd09385-9d78-428c-bb46-efeb37e8e55a\") " pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:32.725869 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:32.725341    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 01:12:32.725869 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:32.725403    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cd09385-9d78-428c-bb46-efeb37e8e55a-original-pull-secret podName:2cd09385-9d78-428c-bb46-efeb37e8e55a nodeName:}" failed. No retries permitted until 2026-09-19 01:12:36.725385142 +0000 UTC m=+11.632247174 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2cd09385-9d78-428c-bb46-efeb37e8e55a-original-pull-secret") pod "global-pull-secret-syncer-kj96s" (UID: "2cd09385-9d78-428c-bb46-efeb37e8e55a") : object "kube-system"/"original-pull-secret" not registered
Sep 19 01:12:32.725869 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:32.725473    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 01:12:32.725869 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:32.725526    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01b79542-9c56-4f26-a1d4-c4f2257ff152-metrics-certs podName:01b79542-9c56-4f26-a1d4-c4f2257ff152 nodeName:}" failed. No retries permitted until 2026-09-19 01:12:36.725512962 +0000 UTC m=+11.632375007 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/01b79542-9c56-4f26-a1d4-c4f2257ff152-metrics-certs") pod "network-metrics-daemon-hpjz8" (UID: "01b79542-9c56-4f26-a1d4-c4f2257ff152") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 01:12:32.826426 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:32.826389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77q5s\" (UniqueName: \"kubernetes.io/projected/538e796f-8768-4bf9-ab1c-c3631bb14317-kube-api-access-77q5s\") pod \"network-check-target-nh6vq\" (UID: \"538e796f-8768-4bf9-ab1c-c3631bb14317\") " pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:12:32.827225 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:32.826713    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 01:12:32.827225 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:32.826742    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 01:12:32.827225 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:32.826755    2580 projected.go:196] Error preparing data for projected volume kube-api-access-77q5s for pod openshift-network-diagnostics/network-check-target-nh6vq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 01:12:32.827225 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:32.826816    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/538e796f-8768-4bf9-ab1c-c3631bb14317-kube-api-access-77q5s podName:538e796f-8768-4bf9-ab1c-c3631bb14317 nodeName:}" failed. No retries permitted until 2026-09-19 01:12:36.826798607 +0000 UTC m=+11.733660646 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-77q5s" (UniqueName: "kubernetes.io/projected/538e796f-8768-4bf9-ab1c-c3631bb14317-kube-api-access-77q5s") pod "network-check-target-nh6vq" (UID: "538e796f-8768-4bf9-ab1c-c3631bb14317") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 01:12:33.059638 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:33.058860    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:33.059638 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:33.058988    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj96s" podUID="2cd09385-9d78-428c-bb46-efeb37e8e55a"
Sep 19 01:12:33.059638 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:33.059357    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:12:33.059638 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:33.059441    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh6vq" podUID="538e796f-8768-4bf9-ab1c-c3631bb14317"
Sep 19 01:12:33.059638 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:33.059510    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:33.059638 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:33.059593    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hpjz8" podUID="01b79542-9c56-4f26-a1d4-c4f2257ff152"
Sep 19 01:12:35.057419 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:35.057382    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:35.057959 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:35.057528    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj96s" podUID="2cd09385-9d78-428c-bb46-efeb37e8e55a"
Sep 19 01:12:35.057959 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:35.057382    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:12:35.058062 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:35.057980    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh6vq" podUID="538e796f-8768-4bf9-ab1c-c3631bb14317"
Sep 19 01:12:35.058062 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:35.058034    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:35.058159 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:35.058140    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hpjz8" podUID="01b79542-9c56-4f26-a1d4-c4f2257ff152"
Sep 19 01:12:36.759472 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:36.759440    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2cd09385-9d78-428c-bb46-efeb37e8e55a-original-pull-secret\") pod \"global-pull-secret-syncer-kj96s\" (UID: \"2cd09385-9d78-428c-bb46-efeb37e8e55a\") " pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:36.759894 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:36.759537    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01b79542-9c56-4f26-a1d4-c4f2257ff152-metrics-certs\") pod \"network-metrics-daemon-hpjz8\" (UID: \"01b79542-9c56-4f26-a1d4-c4f2257ff152\") " pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:36.759894 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:36.759589    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 01:12:36.759894 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:36.759643    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 01:12:36.759894 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:36.759663    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cd09385-9d78-428c-bb46-efeb37e8e55a-original-pull-secret podName:2cd09385-9d78-428c-bb46-efeb37e8e55a nodeName:}" failed. No retries permitted until 2026-09-19 01:12:44.759645231 +0000 UTC m=+19.666507259 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2cd09385-9d78-428c-bb46-efeb37e8e55a-original-pull-secret") pod "global-pull-secret-syncer-kj96s" (UID: "2cd09385-9d78-428c-bb46-efeb37e8e55a") : object "kube-system"/"original-pull-secret" not registered
Sep 19 01:12:36.759894 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:36.759693    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01b79542-9c56-4f26-a1d4-c4f2257ff152-metrics-certs podName:01b79542-9c56-4f26-a1d4-c4f2257ff152 nodeName:}" failed. No retries permitted until 2026-09-19 01:12:44.759677955 +0000 UTC m=+19.666539990 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/01b79542-9c56-4f26-a1d4-c4f2257ff152-metrics-certs") pod "network-metrics-daemon-hpjz8" (UID: "01b79542-9c56-4f26-a1d4-c4f2257ff152") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 01:12:36.860373 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:36.860336    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77q5s\" (UniqueName: \"kubernetes.io/projected/538e796f-8768-4bf9-ab1c-c3631bb14317-kube-api-access-77q5s\") pod \"network-check-target-nh6vq\" (UID: \"538e796f-8768-4bf9-ab1c-c3631bb14317\") " pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:12:36.860533 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:36.860475    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 01:12:36.860533 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:36.860510    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 01:12:36.860533 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:36.860522    2580 projected.go:196] Error preparing data for projected volume kube-api-access-77q5s for pod openshift-network-diagnostics/network-check-target-nh6vq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 01:12:36.860647 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:36.860575    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/538e796f-8768-4bf9-ab1c-c3631bb14317-kube-api-access-77q5s podName:538e796f-8768-4bf9-ab1c-c3631bb14317 nodeName:}" failed. No retries permitted until 2026-09-19 01:12:44.860557975 +0000 UTC m=+19.767420001 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-77q5s" (UniqueName: "kubernetes.io/projected/538e796f-8768-4bf9-ab1c-c3631bb14317-kube-api-access-77q5s") pod "network-check-target-nh6vq" (UID: "538e796f-8768-4bf9-ab1c-c3631bb14317") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 01:12:37.063683 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:37.063605    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:37.063827 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:37.063783    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hpjz8" podUID="01b79542-9c56-4f26-a1d4-c4f2257ff152"
Sep 19 01:12:37.063827 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:37.063790    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:37.063827 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:37.063816    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:12:37.063981 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:37.063878    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj96s" podUID="2cd09385-9d78-428c-bb46-efeb37e8e55a"
Sep 19 01:12:37.063981 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:37.063943    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh6vq" podUID="538e796f-8768-4bf9-ab1c-c3631bb14317"
Sep 19 01:12:39.062845 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:39.062812    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:39.063282 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:39.063057    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hpjz8" podUID="01b79542-9c56-4f26-a1d4-c4f2257ff152"
Sep 19 01:12:39.064019 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:39.063908    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:39.064792 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:39.064713    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj96s" podUID="2cd09385-9d78-428c-bb46-efeb37e8e55a"
Sep 19 01:12:39.064792 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:39.064784    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:12:39.064916 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:39.064861    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh6vq" podUID="538e796f-8768-4bf9-ab1c-c3631bb14317"
Sep 19 01:12:41.057025 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:41.056993    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:41.057531 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:41.057030    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:41.057531 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:41.056993    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:12:41.057531 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:41.057124    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj96s" podUID="2cd09385-9d78-428c-bb46-efeb37e8e55a"
Sep 19 01:12:41.057531 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:41.057223    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh6vq" podUID="538e796f-8768-4bf9-ab1c-c3631bb14317"
Sep 19 01:12:41.057531 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:41.057307    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hpjz8" podUID="01b79542-9c56-4f26-a1d4-c4f2257ff152"
Sep 19 01:12:43.057224 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:43.057139    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:12:43.057730 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:43.057157    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:43.057730 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:43.057269    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh6vq" podUID="538e796f-8768-4bf9-ab1c-c3631bb14317"
Sep 19 01:12:43.057730 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:43.057284    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:43.057730 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:43.057382    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj96s" podUID="2cd09385-9d78-428c-bb46-efeb37e8e55a"
Sep 19 01:12:43.057730 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:43.057484    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hpjz8" podUID="01b79542-9c56-4f26-a1d4-c4f2257ff152"
Sep 19 01:12:44.817584 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:44.817490    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2cd09385-9d78-428c-bb46-efeb37e8e55a-original-pull-secret\") pod \"global-pull-secret-syncer-kj96s\" (UID: \"2cd09385-9d78-428c-bb46-efeb37e8e55a\") " pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:44.817992 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:44.817602    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 01:12:44.817992 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:44.817673    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cd09385-9d78-428c-bb46-efeb37e8e55a-original-pull-secret podName:2cd09385-9d78-428c-bb46-efeb37e8e55a nodeName:}" failed. No retries permitted until 2026-09-19 01:13:00.817655153 +0000 UTC m=+35.724517203 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2cd09385-9d78-428c-bb46-efeb37e8e55a-original-pull-secret") pod "global-pull-secret-syncer-kj96s" (UID: "2cd09385-9d78-428c-bb46-efeb37e8e55a") : object "kube-system"/"original-pull-secret" not registered
Sep 19 01:12:44.817992 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:44.817696    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01b79542-9c56-4f26-a1d4-c4f2257ff152-metrics-certs\") pod \"network-metrics-daemon-hpjz8\" (UID: \"01b79542-9c56-4f26-a1d4-c4f2257ff152\") " pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:44.817992 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:44.817792    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 01:12:44.817992 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:44.817832    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01b79542-9c56-4f26-a1d4-c4f2257ff152-metrics-certs podName:01b79542-9c56-4f26-a1d4-c4f2257ff152 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:00.817819674 +0000 UTC m=+35.724681701 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/01b79542-9c56-4f26-a1d4-c4f2257ff152-metrics-certs") pod "network-metrics-daemon-hpjz8" (UID: "01b79542-9c56-4f26-a1d4-c4f2257ff152") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 01:12:44.918556 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:44.918519    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77q5s\" (UniqueName: \"kubernetes.io/projected/538e796f-8768-4bf9-ab1c-c3631bb14317-kube-api-access-77q5s\") pod \"network-check-target-nh6vq\" (UID: \"538e796f-8768-4bf9-ab1c-c3631bb14317\") " pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:12:44.918697 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:44.918680    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 01:12:44.918779 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:44.918704    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 01:12:44.918779 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:44.918717    2580 projected.go:196] Error preparing data for projected volume kube-api-access-77q5s for pod openshift-network-diagnostics/network-check-target-nh6vq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 01:12:44.918779 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:44.918773    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/538e796f-8768-4bf9-ab1c-c3631bb14317-kube-api-access-77q5s podName:538e796f-8768-4bf9-ab1c-c3631bb14317 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:00.918755435 +0000 UTC m=+35.825617461 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-77q5s" (UniqueName: "kubernetes.io/projected/538e796f-8768-4bf9-ab1c-c3631bb14317-kube-api-access-77q5s") pod "network-check-target-nh6vq" (UID: "538e796f-8768-4bf9-ab1c-c3631bb14317") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 01:12:45.057213 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:45.057176    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:12:45.057213 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:45.057206    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:45.057447 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:45.057180    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:45.057447 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:45.057321    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh6vq" podUID="538e796f-8768-4bf9-ab1c-c3631bb14317"
Sep 19 01:12:45.057715 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:45.057679    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hpjz8" podUID="01b79542-9c56-4f26-a1d4-c4f2257ff152"
Sep 19 01:12:45.057826 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:45.057770    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj96s" podUID="2cd09385-9d78-428c-bb46-efeb37e8e55a"
Sep 19 01:12:47.058144 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:47.058115    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:47.058609 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:47.058215    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj96s" podUID="2cd09385-9d78-428c-bb46-efeb37e8e55a"
Sep 19 01:12:47.058609 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:47.058310    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:47.058609 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:47.058461    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hpjz8" podUID="01b79542-9c56-4f26-a1d4-c4f2257ff152"
Sep 19 01:12:47.058609 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:47.058527    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:12:47.058609 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:47.058600    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh6vq" podUID="538e796f-8768-4bf9-ab1c-c3631bb14317"
Sep 19 01:12:48.229427 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:48.229200    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-grkk4" event={"ID":"54557d74-f7ed-44ac-b483-9942019a9c4d","Type":"ContainerStarted","Data":"43a1c7f969c17e7ad723a96b58c400f872c6e1d04d533f36b60162bc2d4a2f8c"}
Sep 19 01:12:48.231066 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:48.231045    2580 generic.go:356] "Generic (PLEG): container finished" podID="23fcc30f-348f-4c04-aff2-23fe3420219a" containerID="2da24c5ef3ba235e4fb4315e28442c63fffeced3cb5b2b2fb061b2cf404cd552" exitCode=0
Sep 19 01:12:48.231154 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:48.231131    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tfq72" event={"ID":"23fcc30f-348f-4c04-aff2-23fe3420219a","Type":"ContainerDied","Data":"2da24c5ef3ba235e4fb4315e28442c63fffeced3cb5b2b2fb061b2cf404cd552"}
Sep 19 01:12:48.232767 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:48.232734    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dvd6m" event={"ID":"2857e44c-032d-4586-b438-26ad712aef9c","Type":"ContainerStarted","Data":"1660d8eb271303a20da0a9ea29976266d30f6fc14412561b16680ae7ad8d00b3"}
Sep 19 01:12:48.234234 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:48.234217    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-46sks" event={"ID":"f6894680-df25-4249-b9ef-c79e4bebdd3e","Type":"ContainerStarted","Data":"d16a876d60aa7a7d3bfb661f0f6f566dc5f87947246535e881b8d1840264a7a8"}
Sep 19 01:12:48.237578 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:48.236319    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9n69k" event={"ID":"2f997c70-10b2-49e0-8711-00e914baa6c4","Type":"ContainerStarted","Data":"b5373f5d8622bf044c4bac804982e9342e1986fd1c24b2d1ea1f7e6d25bed110"}
Sep 19 01:12:48.240693 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:48.240674    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7z77w" event={"ID":"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4","Type":"ContainerStarted","Data":"03b68267e53e403ab3d3e2eebf7d2e1c1cd48bbc9c01d4f1c5071dfa5558eb85"}
Sep 19 01:12:48.240790 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:48.240741    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7z77w" event={"ID":"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4","Type":"ContainerStarted","Data":"37acc23b5f71aa37542f9277c3dbfee6f97439061995da874e488c6ca0c75313"}
Sep 19 01:12:48.240846 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:48.240791    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7z77w" event={"ID":"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4","Type":"ContainerStarted","Data":"25e851d59bf7c0224f860148c68d752dd88500f4fc8a50f0310227ffefa0a802"}
Sep 19 01:12:48.240846 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:48.240840    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7z77w" event={"ID":"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4","Type":"ContainerStarted","Data":"ec4321147461ec10a51e4a1bd4fc37f1132b14dc72b4ed3120376bfacceeee43"}
Sep 19 01:12:48.240949 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:48.240886    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7z77w" event={"ID":"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4","Type":"ContainerStarted","Data":"81ffc305838ae8e5d9a60adcae06219cfb5438af08ac07f5d214bde2292ce255"}
Sep 19 01:12:48.240949 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:48.240934    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7z77w" event={"ID":"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4","Type":"ContainerStarted","Data":"6c68080f81e1bca4ed98992a5c4408d2ede90b22a629ce7cca6668e31bab0b2b"}
Sep 19 01:12:48.241984 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:48.241956    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk" event={"ID":"da7b0d9d-cbe8-4f97-87e0-806003cd917b","Type":"ContainerStarted","Data":"99669b33e5092c1e091ed6d51273528ab2054160d6ed22b6ec77663ad89fba21"}
Sep 19 01:12:48.243176 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:48.243158    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d4nk2" event={"ID":"3347022a-f11e-4bad-a7dd-902cf5613aa5","Type":"ContainerStarted","Data":"236cd0c73b494b4e111d66e3b2eb624cc353f11053c7503b7386dcd694d52f07"}
Sep 19 01:12:48.243985 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:48.243949    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-grkk4" podStartSLOduration=3.979222434 podStartE2EDuration="21.243938398s" totalImagesPullingTime="17.264715964s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 01:12:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:12:48.243470751 +0000 UTC m=+23.150332799" watchObservedRunningTime="2026-09-19 01:12:48.243938398 +0000 UTC m=+23.150800445"
Sep 19 01:12:48.244060 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:48.244013    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-76.ec2.internal" podStartSLOduration=20.244008924 podStartE2EDuration="20.244008924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 01:12:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:12:32.230681585 +0000 UTC m=+7.137543648" watchObservedRunningTime="2026-09-19 01:12:48.244008924 +0000 UTC m=+23.150870973"
Sep 19 01:12:48.280340 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:48.280291    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-d4nk2" podStartSLOduration=3.837315014 podStartE2EDuration="21.280276354s" totalImagesPullingTime="17.44296134s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 01:12:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:12:48.280208938 +0000 UTC m=+23.187070987" watchObservedRunningTime="2026-09-19 01:12:48.280276354 +0000 UTC m=+23.187138404"
Sep 19 01:12:48.313623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:48.313587    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9n69k" podStartSLOduration=4.054278709 podStartE2EDuration="21.313575798s" totalImagesPullingTime="17.259297089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:12:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:12:48.313233402 +0000 UTC m=+23.220095449" watchObservedRunningTime="2026-09-19 01:12:48.313575798 +0000 UTC m=+23.220437845"
Sep 19 01:12:48.313677 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:48.313655    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-46sks" podStartSLOduration=4.056701838 podStartE2EDuration="21.313651449s" totalImagesPullingTime="17.256949611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 01:12:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:12:48.297315686 +0000 UTC m=+23.204177734" watchObservedRunningTime="2026-09-19 01:12:48.313651449 +0000 UTC m=+23.220513500"
Sep 19 01:12:48.338084 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:48.338053    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dvd6m" podStartSLOduration=7.957732889 podStartE2EDuration="21.338043367s" totalImagesPullingTime="13.380310478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 01:12:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:12:48.338016031 +0000 UTC m=+23.244878078" watchObservedRunningTime="2026-09-19 01:12:48.338043367 +0000 UTC m=+23.244905415"
Sep 19 01:12:49.057513 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:49.057471    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:49.057681 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:49.057520    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:12:49.057681 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:49.057632    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hpjz8" podUID="01b79542-9c56-4f26-a1d4-c4f2257ff152"
Sep 19 01:12:49.057681 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:49.057671    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:49.057803 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:49.057770    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj96s" podUID="2cd09385-9d78-428c-bb46-efeb37e8e55a"
Sep 19 01:12:49.057869 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:49.057848    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh6vq" podUID="538e796f-8768-4bf9-ab1c-c3631bb14317"
Sep 19 01:12:49.246981 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:49.246947    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j6q6r" event={"ID":"56a6eadc-607c-4b80-8dc8-183e63551d8b","Type":"ContainerStarted","Data":"b3b35d57f2e59bb9c4223c2c88a64bfddf6df628b89e6af49468b69aa49cb21e"}
Sep 19 01:12:49.282609 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:49.282568    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-j6q6r" podStartSLOduration=5.024184881 podStartE2EDuration="22.282554023s" totalImagesPullingTime="17.258369142s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 01:12:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:12:49.282458343 +0000 UTC m=+24.189320382" watchObservedRunningTime="2026-09-19 01:12:49.282554023 +0000 UTC m=+24.189416117"
Sep 19 01:12:49.348717 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:49.348687    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 19 01:12:50.066775 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:50.066635    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-19T01:12:49.348705973Z","UUID":"a0d76de4-eed1-4915-b0ea-7682b0521237","Handler":null,"Name":"","Endpoint":""}
Sep 19 01:12:50.068345 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:50.068318    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 19 01:12:50.068475 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:50.068353    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 19 01:12:50.251929 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:50.251890    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk" event={"ID":"da7b0d9d-cbe8-4f97-87e0-806003cd917b","Type":"ContainerStarted","Data":"d48d087abf40ef22b442a6ab8f42f2612cd4b25f34cdd8aca3ff4ff25f46945f"}
Sep 19 01:12:51.057040 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:51.057003    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:51.057190 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:51.057124    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj96s" podUID="2cd09385-9d78-428c-bb46-efeb37e8e55a"
Sep 19 01:12:51.057190 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:51.057008    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:12:51.057314 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:51.057217    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh6vq" podUID="538e796f-8768-4bf9-ab1c-c3631bb14317"
Sep 19 01:12:51.057314 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:51.057239    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:51.057402 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:51.057379    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hpjz8" podUID="01b79542-9c56-4f26-a1d4-c4f2257ff152"
Sep 19 01:12:51.257050 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:51.257010    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk" event={"ID":"da7b0d9d-cbe8-4f97-87e0-806003cd917b","Type":"ContainerStarted","Data":"e7476f3b66ee7769d813e8ebd9daac718f694749ff48d4587c624e3a7aeea9da"}
Sep 19 01:12:51.260960 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:51.260922    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7z77w" event={"ID":"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4","Type":"ContainerStarted","Data":"e58f506421f63c1ff786f09d9479df474d6b88ac1d4dcb8bbc440550f9989d36"}
Sep 19 01:12:51.275564 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:51.275526    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wqmk" podStartSLOduration=4.711609249 podStartE2EDuration="24.275514214s" totalImagesPullingTime="19.563904965s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 01:12:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:12:51.27493752 +0000 UTC m=+26.181799568" watchObservedRunningTime="2026-09-19 01:12:51.275514214 +0000 UTC m=+26.182376260"
Sep 19 01:12:51.279065 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:51.279048    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9n69k"
Sep 19 01:12:51.279553 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:51.279538    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9n69k"
Sep 19 01:12:52.263754 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:52.263715    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9n69k"
Sep 19 01:12:52.264244 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:52.264151    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9n69k"
Sep 19 01:12:53.058754 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:53.058535    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:53.058873 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:53.058567    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:12:53.058959 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:53.058865    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj96s" podUID="2cd09385-9d78-428c-bb46-efeb37e8e55a"
Sep 19 01:12:53.058959 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:53.058644    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:53.058959 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:53.058938    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh6vq" podUID="538e796f-8768-4bf9-ab1c-c3631bb14317"
Sep 19 01:12:53.059091 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:53.059064    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hpjz8" podUID="01b79542-9c56-4f26-a1d4-c4f2257ff152"
Sep 19 01:12:53.269239 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:53.269214    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7z77w" event={"ID":"888ee4fc-23e7-4ee4-ac81-dd9f21a09fb4","Type":"ContainerStarted","Data":"c6452c02f71acc8dfc88103fd17b43b358088cd0ab5c4718232edcc3af8c1f50"}
Sep 19 01:12:53.269789 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:53.269481    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:53.271459 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:53.271438    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tfq72" event={"ID":"23fcc30f-348f-4c04-aff2-23fe3420219a","Type":"ContainerStarted","Data":"89d627f36b49d4f61ca5df5b227dc2be18f00834ee1e4d563a80c2016c6a14dd"}
Sep 19 01:12:53.284336 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:53.284317    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:53.299662 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:53.299626    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7z77w" podStartSLOduration=8.932597705 podStartE2EDuration="26.299613324s" totalImagesPullingTime="17.367015619s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 01:12:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:12:53.299184412 +0000 UTC m=+28.206046494" watchObservedRunningTime="2026-09-19 01:12:53.299613324 +0000 UTC m=+28.206475369"
Sep 19 01:12:54.275247 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:54.275209    2580 generic.go:356] "Generic (PLEG): container finished" podID="23fcc30f-348f-4c04-aff2-23fe3420219a" containerID="89d627f36b49d4f61ca5df5b227dc2be18f00834ee1e4d563a80c2016c6a14dd" exitCode=0
Sep 19 01:12:54.275673 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:54.275295    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tfq72" event={"ID":"23fcc30f-348f-4c04-aff2-23fe3420219a","Type":"ContainerDied","Data":"89d627f36b49d4f61ca5df5b227dc2be18f00834ee1e4d563a80c2016c6a14dd"}
Sep 19 01:12:54.275673 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:54.275651    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:54.276410 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:54.275989    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:54.291190 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:54.291172    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:12:54.908894 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:54.908861    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kj96s"]
Sep 19 01:12:54.909049 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:54.908998    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:54.909095 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:54.909078    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj96s" podUID="2cd09385-9d78-428c-bb46-efeb37e8e55a"
Sep 19 01:12:54.912165 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:54.912137    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nh6vq"]
Sep 19 01:12:54.912320 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:54.912236    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:12:54.912320 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:54.912313    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh6vq" podUID="538e796f-8768-4bf9-ab1c-c3631bb14317"
Sep 19 01:12:54.912686 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:54.912664    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hpjz8"]
Sep 19 01:12:54.912765 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:54.912753    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:54.912877 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:54.912857    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hpjz8" podUID="01b79542-9c56-4f26-a1d4-c4f2257ff152"
Sep 19 01:12:56.056783 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:56.056750    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:56.057338 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:56.056859    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hpjz8" podUID="01b79542-9c56-4f26-a1d4-c4f2257ff152"
Sep 19 01:12:56.057338 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:56.056888    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:56.057338 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:56.056938    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj96s" podUID="2cd09385-9d78-428c-bb46-efeb37e8e55a"
Sep 19 01:12:56.281136 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:56.281102    2580 generic.go:356] "Generic (PLEG): container finished" podID="23fcc30f-348f-4c04-aff2-23fe3420219a" containerID="f9fb4c989cfc8ddd618f6932f67e679c5288eeca7428f5ce28870d57301464fa" exitCode=0
Sep 19 01:12:56.281306 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:56.281186    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tfq72" event={"ID":"23fcc30f-348f-4c04-aff2-23fe3420219a","Type":"ContainerDied","Data":"f9fb4c989cfc8ddd618f6932f67e679c5288eeca7428f5ce28870d57301464fa"}
Sep 19 01:12:57.059542 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:57.059510    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:12:57.060020 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:57.059623    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh6vq" podUID="538e796f-8768-4bf9-ab1c-c3631bb14317"
Sep 19 01:12:57.286311 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:57.286096    2580 generic.go:356] "Generic (PLEG): container finished" podID="23fcc30f-348f-4c04-aff2-23fe3420219a" containerID="47029c918e81dd88eb69e171dae9fa658c3da475f97b736b4873dedd4612ef88" exitCode=0
Sep 19 01:12:57.286433 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:57.286169    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tfq72" event={"ID":"23fcc30f-348f-4c04-aff2-23fe3420219a","Type":"ContainerDied","Data":"47029c918e81dd88eb69e171dae9fa658c3da475f97b736b4873dedd4612ef88"}
Sep 19 01:12:58.057022 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:58.056990    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:12:58.057216 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:58.056991    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:12:58.057216 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:58.057118    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hpjz8" podUID="01b79542-9c56-4f26-a1d4-c4f2257ff152"
Sep 19 01:12:58.057216 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:58.057192    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj96s" podUID="2cd09385-9d78-428c-bb46-efeb37e8e55a"
Sep 19 01:12:59.056832 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:12:59.056792    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:12:59.057186 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:12:59.056918    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh6vq" podUID="538e796f-8768-4bf9-ab1c-c3631bb14317"
Sep 19 01:13:00.057108 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.057073    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:13:00.057723 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:00.057209    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hpjz8" podUID="01b79542-9c56-4f26-a1d4-c4f2257ff152"
Sep 19 01:13:00.057723 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.057251    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:13:00.057723 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:00.057338    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kj96s" podUID="2cd09385-9d78-428c-bb46-efeb37e8e55a"
Sep 19 01:13:00.635807 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.635779    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-76.ec2.internal" event="NodeReady"
Sep 19 01:13:00.636015 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.635928    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 19 01:13:00.683339 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.683310    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"]
Sep 19 01:13:00.702432 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.702403    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-p6l4m"]
Sep 19 01:13:00.702592 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.702531    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:00.705241 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.705213    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 19 01:13:00.705551 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.705533    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5r66s\""
Sep 19 01:13:00.705625 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.705548    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 19 01:13:00.705975 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.705952    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 19 01:13:00.712881 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.712857    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 19 01:13:00.714526 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.714491    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-zmkjg"]
Sep 19 01:13:00.714653 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.714635    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m"
Sep 19 01:13:00.720041 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.719513    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 19 01:13:00.720041 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.719772    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 19 01:13:00.724578 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.724557    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 19 01:13:00.724771 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.724740    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 19 01:13:00.724878 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.724804    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-84czs\""
Sep 19 01:13:00.728051 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.728030    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 19 01:13:00.730680 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.730664    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"]
Sep 19 01:13:00.730769 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.730694    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-zmkjg"]
Sep 19 01:13:00.730769 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.730719    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-v6ppc"]
Sep 19 01:13:00.730880 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.730799    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-zmkjg"
Sep 19 01:13:00.736479 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.736430    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 19 01:13:00.736716 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.736701    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 19 01:13:00.736935 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.736913    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-8qnqw\""
Sep 19 01:13:00.742671 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.742652    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-p6l4m"]
Sep 19 01:13:00.742748 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.742675    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v6ppc"
Sep 19 01:13:00.746071 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.746045    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 19 01:13:00.746186 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.746168    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h9l7c\""
Sep 19 01:13:00.746266 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.746249    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 19 01:13:00.751964 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.749351    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v6ppc"]
Sep 19 01:13:00.755766 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.755741    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-44vdn"]
Sep 19 01:13:00.777884 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.777858    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-f6bdh"]
Sep 19 01:13:00.778034 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.777996    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-44vdn"
Sep 19 01:13:00.781471 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.781449    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 19 01:13:00.782274 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.782260    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 19 01:13:00.782961 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.782922    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 19 01:13:00.783256 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.783234    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-96s5w\""
Sep 19 01:13:00.783466 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.783436    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 19 01:13:00.793873 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.793856    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-f5gtc"]
Sep 19 01:13:00.793977 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.793961    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh"
Sep 19 01:13:00.797838 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.797523    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 19 01:13:00.797838 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.797703    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 19 01:13:00.797838 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.797811    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 19 01:13:00.798043 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.797870    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-nbj65\""
Sep 19 01:13:00.798043 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.797919    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 19 01:13:00.805440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.805417    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 19 01:13:00.812055 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.812039    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-bvc7f"]
Sep 19 01:13:00.812193 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.812177    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-f5gtc"
Sep 19 01:13:00.814693 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.814674    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-4rbzb\""
Sep 19 01:13:00.814791 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.814702    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 19 01:13:00.814853 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.814811    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 19 01:13:00.815144 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.815127    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 19 01:13:00.825613 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.825592    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39355481-6d16-4553-961a-f03ca171e712-tmp\") pod \"insights-operator-6f9c644dd9-p6l4m\" (UID: \"39355481-6d16-4553-961a-f03ca171e712\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m"
Sep 19 01:13:00.825708 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.825625    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a439b5fa-a142-4815-b145-c3278eea25a0-ca-trust-extracted\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:00.825708 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.825667    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/39355481-6d16-4553-961a-f03ca171e712-serving-cert\") pod \"insights-operator-6f9c644dd9-p6l4m\" (UID: \"39355481-6d16-4553-961a-f03ca171e712\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m"
Sep 19 01:13:00.825708 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.825688    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a439b5fa-a142-4815-b145-c3278eea25a0-installation-pull-secrets\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:00.825867 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.825739    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39355481-6d16-4553-961a-f03ca171e712-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-p6l4m\" (UID: \"39355481-6d16-4553-961a-f03ca171e712\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m"
Sep 19 01:13:00.825867 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.825766    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/39355481-6d16-4553-961a-f03ca171e712-snapshots\") pod \"insights-operator-6f9c644dd9-p6l4m\" (UID: \"39355481-6d16-4553-961a-f03ca171e712\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m"
Sep 19 01:13:00.825867 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.825819    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a439b5fa-a142-4815-b145-c3278eea25a0-registry-certificates\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:00.825867 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.825843    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-bound-sa-token\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:00.826066 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.825869    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f921d091-7bf8-4d6d-81e4-6162afac0198-config-volume\") pod \"dns-default-v6ppc\" (UID: \"f921d091-7bf8-4d6d-81e4-6162afac0198\") " pod="openshift-dns/dns-default-v6ppc"
Sep 19 01:13:00.826066 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.825909    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-registry-tls\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:00.826066 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.825930    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39355481-6d16-4553-961a-f03ca171e712-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-p6l4m\" (UID: \"39355481-6d16-4553-961a-f03ca171e712\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m"
Sep 19 01:13:00.826066 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.825946    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6t7t\" (UniqueName: \"kubernetes.io/projected/73e4b372-92e4-4eeb-ac37-6c5ec11217f0-kube-api-access-j6t7t\") pod \"volume-data-source-validator-5c4db8bdf9-zmkjg\" (UID: \"73e4b372-92e4-4eeb-ac37-6c5ec11217f0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-zmkjg"
Sep 19 01:13:00.826066 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.825964    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f921d091-7bf8-4d6d-81e4-6162afac0198-metrics-tls\") pod \"dns-default-v6ppc\" (UID: \"f921d091-7bf8-4d6d-81e4-6162afac0198\") " pod="openshift-dns/dns-default-v6ppc"
Sep 19 01:13:00.826066 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.825979    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f921d091-7bf8-4d6d-81e4-6162afac0198-tmp-dir\") pod \"dns-default-v6ppc\" (UID: \"f921d091-7bf8-4d6d-81e4-6162afac0198\") " pod="openshift-dns/dns-default-v6ppc"
Sep 19 01:13:00.826066 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.826002    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01b79542-9c56-4f26-a1d4-c4f2257ff152-metrics-certs\") pod \"network-metrics-daemon-hpjz8\" (UID: \"01b79542-9c56-4f26-a1d4-c4f2257ff152\") " pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:13:00.826066 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.826016    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bgf4\" (UniqueName: \"kubernetes.io/projected/f921d091-7bf8-4d6d-81e4-6162afac0198-kube-api-access-2bgf4\") pod \"dns-default-v6ppc\" (UID: \"f921d091-7bf8-4d6d-81e4-6162afac0198\") " pod="openshift-dns/dns-default-v6ppc"
Sep 19 01:13:00.826066 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.826042    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a439b5fa-a142-4815-b145-c3278eea25a0-image-registry-private-configuration\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:00.826066 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.826068    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a439b5fa-a142-4815-b145-c3278eea25a0-trusted-ca\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:00.826559 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.826097    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz7qc\" (UniqueName: \"kubernetes.io/projected/39355481-6d16-4553-961a-f03ca171e712-kube-api-access-jz7qc\") pod \"insights-operator-6f9c644dd9-p6l4m\" (UID: \"39355481-6d16-4553-961a-f03ca171e712\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m"
Sep 19 01:13:00.826559 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:00.826105    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 01:13:00.826559 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.826129    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2cd09385-9d78-428c-bb46-efeb37e8e55a-original-pull-secret\") pod \"global-pull-secret-syncer-kj96s\" (UID: \"2cd09385-9d78-428c-bb46-efeb37e8e55a\") " pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:13:00.826559 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:00.826153    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01b79542-9c56-4f26-a1d4-c4f2257ff152-metrics-certs podName:01b79542-9c56-4f26-a1d4-c4f2257ff152 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:32.826135507 +0000 UTC m=+67.732997568 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/01b79542-9c56-4f26-a1d4-c4f2257ff152-metrics-certs") pod "network-metrics-daemon-hpjz8" (UID: "01b79542-9c56-4f26-a1d4-c4f2257ff152") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 01:13:00.826559 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.826181    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fg64\" (UniqueName: \"kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-kube-api-access-6fg64\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:00.826559 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:00.826232    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 01:13:00.826559 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:00.826305    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cd09385-9d78-428c-bb46-efeb37e8e55a-original-pull-secret podName:2cd09385-9d78-428c-bb46-efeb37e8e55a nodeName:}" failed. No retries permitted until 2026-09-19 01:13:32.826286133 +0000 UTC m=+67.733148169 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2cd09385-9d78-428c-bb46-efeb37e8e55a-original-pull-secret") pod "global-pull-secret-syncer-kj96s" (UID: "2cd09385-9d78-428c-bb46-efeb37e8e55a") : object "kube-system"/"original-pull-secret" not registered
Sep 19 01:13:00.838120 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.838101    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9bmsn"]
Sep 19 01:13:00.838237 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.838223    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-bvc7f"
Sep 19 01:13:00.841014 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.840990    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 19 01:13:00.841014 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.840999    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-xd6cx\""
Sep 19 01:13:00.841340 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.841321    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 19 01:13:00.841462 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.841416    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 19 01:13:00.841716 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.841669    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 19 01:13:00.858311 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.858291    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9bmsn"
Sep 19 01:13:00.858414 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.858297    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-p4cpb"]
Sep 19 01:13:00.861804 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.861753    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 19 01:13:00.861907 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.861891    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 19 01:13:00.863479 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.862887    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-wbbj7\""
Sep 19 01:13:00.865515 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.863770    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 19 01:13:00.865515 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.864829    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 19 01:13:00.877467 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.877450    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-qxlvd"]
Sep 19 01:13:00.877606 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.877587    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p4cpb"
Sep 19 01:13:00.879880 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.879860    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-spc29\""
Sep 19 01:13:00.879971 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.879908    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 19 01:13:00.880154 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.880130    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 19 01:13:00.880576 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.880561    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 19 01:13:00.890391 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.890373    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-69d68c9d66-2gblx"]
Sep 19 01:13:00.890480 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.890455    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-qxlvd"
Sep 19 01:13:00.893703 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.893646    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-b7n9w\""
Sep 19 01:13:00.893798 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.893730    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 19 01:13:00.894160 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.894132    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 19 01:13:00.911363 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.910547    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-f6bdh"]
Sep 19 01:13:00.911363 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.910581    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9bmsn"]
Sep 19 01:13:00.911363 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.910602    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-bvc7f"]
Sep 19 01:13:00.911363 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.910622    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-f5gtc"]
Sep 19 01:13:00.911363 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.910644    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-44vdn"]
Sep 19 01:13:00.911363 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.910661    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p4cpb"]
Sep 19 01:13:00.911363 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.910684    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-69d68c9d66-2gblx"]
Sep 19 01:13:00.911363 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.910701    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-qxlvd"]
Sep 19 01:13:00.911363 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.910797    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:00.913776 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.913753    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 19 01:13:00.913975 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.913951    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 19 01:13:00.913975 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.913959    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 19 01:13:00.914111 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.914067    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 19 01:13:00.914111 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.914100    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 19 01:13:00.914214 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.914160    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 19 01:13:00.914448 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.914429    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-gsrf7\""
Sep 19 01:13:00.927073 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927049    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39355481-6d16-4553-961a-f03ca171e712-tmp\") pod \"insights-operator-6f9c644dd9-p6l4m\" (UID: \"39355481-6d16-4553-961a-f03ca171e712\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m"
Sep 19 01:13:00.927166 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927088    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/74c8e2f6-1b01-4bd1-b491-ad407a15a109-config\") pod \"console-operator-fd4bb46c-f6bdh\" (UID: \"74c8e2f6-1b01-4bd1-b491-ad407a15a109\") " pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh"
Sep 19 01:13:00.927166 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927116    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/74c8e2f6-1b01-4bd1-b491-ad407a15a109-serving-cert\") pod \"console-operator-fd4bb46c-f6bdh\" (UID: \"74c8e2f6-1b01-4bd1-b491-ad407a15a109\") " pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh"
Sep 19 01:13:00.927166 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927142    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsgkk\" (UniqueName: \"kubernetes.io/projected/2c41153e-4f57-4158-83b8-18563e2750e8-kube-api-access-fsgkk\") pod \"kube-storage-version-migrator-operator-55889f7b47-bvc7f\" (UID: \"2c41153e-4f57-4158-83b8-18563e2750e8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-bvc7f"
Sep 19 01:13:00.927270 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77q5s\" (UniqueName: \"kubernetes.io/projected/538e796f-8768-4bf9-ab1c-c3631bb14317-kube-api-access-77q5s\") pod \"network-check-target-nh6vq\" (UID: \"538e796f-8768-4bf9-ab1c-c3631bb14317\") " pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:13:00.927270 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927202    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c63ea3c5-742f-4077-a661-388d76e45d63-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-9bmsn\" (UID: \"c63ea3c5-742f-4077-a661-388d76e45d63\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9bmsn"
Sep 19 01:13:00.927270 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927228    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2c41153e-4f57-4158-83b8-18563e2750e8-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-bvc7f\" (UID: \"2c41153e-4f57-4158-83b8-18563e2750e8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-bvc7f"
Sep 19 01:13:00.927270 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927256    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/74c8e2f6-1b01-4bd1-b491-ad407a15a109-trusted-ca\") pod \"console-operator-fd4bb46c-f6bdh\" (UID: \"74c8e2f6-1b01-4bd1-b491-ad407a15a109\") " pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh"
Sep 19 01:13:00.927532 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927290    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a439b5fa-a142-4815-b145-c3278eea25a0-installation-pull-secrets\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:00.927532 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927321    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/39355481-6d16-4553-961a-f03ca171e712-snapshots\") pod \"insights-operator-6f9c644dd9-p6l4m\" (UID: \"39355481-6d16-4553-961a-f03ca171e712\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m"
Sep 19 01:13:00.927532 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927338    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qht2p\" (UniqueName: \"kubernetes.io/projected/c63ea3c5-742f-4077-a661-388d76e45d63-kube-api-access-qht2p\") pod \"cluster-monitoring-operator-847b8bd8d7-9bmsn\" (UID: \"c63ea3c5-742f-4077-a661-388d76e45d63\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9bmsn"
Sep 19 01:13:00.927532 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927355    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a439b5fa-a142-4815-b145-c3278eea25a0-registry-certificates\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:00.927532 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927379    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-bound-sa-token\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:00.927532 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927408    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2c41153e-4f57-4158-83b8-18563e2750e8-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-bvc7f\" (UID: \"2c41153e-4f57-4158-83b8-18563e2750e8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-bvc7f"
Sep 19 01:13:00.927532 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927436    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-registry-tls\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:00.927532 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927456    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39355481-6d16-4553-961a-f03ca171e712-tmp\") pod \"insights-operator-6f9c644dd9-p6l4m\" (UID: \"39355481-6d16-4553-961a-f03ca171e712\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m"
Sep 19 01:13:00.927532 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927459    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f921d091-7bf8-4d6d-81e4-6162afac0198-tmp-dir\") pod \"dns-default-v6ppc\" (UID: \"f921d091-7bf8-4d6d-81e4-6162afac0198\") " pod="openshift-dns/dns-default-v6ppc"
Sep 19 01:13:00.927942 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927542    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a439b5fa-a142-4815-b145-c3278eea25a0-image-registry-private-configuration\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:00.927942 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927569    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a439b5fa-a142-4815-b145-c3278eea25a0-trusted-ca\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:00.927942 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927593    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8bdd86a3-9335-4e27-b183-c07df2869fb5-serving-cert\") pod \"service-ca-operator-57cdc4b768-44vdn\" (UID: \"8bdd86a3-9335-4e27-b183-c07df2869fb5\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-44vdn"
Sep 19 01:13:00.927942 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927628    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c63ea3c5-742f-4077-a661-388d76e45d63-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-9bmsn\" (UID: \"c63ea3c5-742f-4077-a661-388d76e45d63\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9bmsn"
Sep 19 01:13:00.927942 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927651    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a439b5fa-a142-4815-b145-c3278eea25a0-ca-trust-extracted\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:00.927942 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927678    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/39355481-6d16-4553-961a-f03ca171e712-serving-cert\") pod \"insights-operator-6f9c644dd9-p6l4m\" (UID: \"39355481-6d16-4553-961a-f03ca171e712\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m"
Sep 19 01:13:00.927942 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927705    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39355481-6d16-4553-961a-f03ca171e712-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-p6l4m\" (UID: \"39355481-6d16-4553-961a-f03ca171e712\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m"
Sep 19 01:13:00.927942 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927729    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq67w\" (UniqueName: \"kubernetes.io/projected/74c8e2f6-1b01-4bd1-b491-ad407a15a109-kube-api-access-kq67w\") pod \"console-operator-fd4bb46c-f6bdh\" (UID: \"74c8e2f6-1b01-4bd1-b491-ad407a15a109\") " pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh"
Sep 19 01:13:00.927942 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927743    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f921d091-7bf8-4d6d-81e4-6162afac0198-tmp-dir\") pod \"dns-default-v6ppc\" (UID: \"f921d091-7bf8-4d6d-81e4-6162afac0198\") " pod="openshift-dns/dns-default-v6ppc"
Sep 19 01:13:00.927942 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927749    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8bdd86a3-9335-4e27-b183-c07df2869fb5-config\") pod \"service-ca-operator-57cdc4b768-44vdn\" (UID: \"8bdd86a3-9335-4e27-b183-c07df2869fb5\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-44vdn"
Sep 19 01:13:00.927942 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927772    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nrsf\" (UniqueName: \"kubernetes.io/projected/af99297d-b9c3-4e8d-b018-cb271f970733-kube-api-access-8nrsf\") pod \"cluster-samples-operator-685d7d68b5-f5gtc\" (UID: \"af99297d-b9c3-4e8d-b018-cb271f970733\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-f5gtc"
Sep 19 01:13:00.927942 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927795    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f921d091-7bf8-4d6d-81e4-6162afac0198-config-volume\") pod \"dns-default-v6ppc\" (UID: \"f921d091-7bf8-4d6d-81e4-6162afac0198\") " pod="openshift-dns/dns-default-v6ppc"
Sep 19 01:13:00.927942 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39355481-6d16-4553-961a-f03ca171e712-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-p6l4m\" (UID: \"39355481-6d16-4553-961a-f03ca171e712\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m"
Sep 19 01:13:00.927942 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927845    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6t7t\" (UniqueName: \"kubernetes.io/projected/73e4b372-92e4-4eeb-ac37-6c5ec11217f0-kube-api-access-j6t7t\") pod \"volume-data-source-validator-5c4db8bdf9-zmkjg\" (UID: \"73e4b372-92e4-4eeb-ac37-6c5ec11217f0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-zmkjg"
Sep 19 01:13:00.927942 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:00.927855    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 01:13:00.927942 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:00.927869    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw: secret "image-registry-tls" not found
Sep 19 01:13:00.927942 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927902    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a439b5fa-a142-4815-b145-c3278eea25a0-registry-certificates\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:00.928684 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:00.927919    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-registry-tls podName:a439b5fa-a142-4815-b145-c3278eea25a0 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:01.427903167 +0000 UTC m=+36.334765197 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-registry-tls") pod "image-registry-5d7bd8c8fd-kxlmw" (UID: "a439b5fa-a142-4815-b145-c3278eea25a0") : secret "image-registry-tls" not found
Sep 19 01:13:00.928684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927919    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/39355481-6d16-4553-961a-f03ca171e712-snapshots\") pod \"insights-operator-6f9c644dd9-p6l4m\" (UID: \"39355481-6d16-4553-961a-f03ca171e712\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m"
Sep 19 01:13:00.928684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927869    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f921d091-7bf8-4d6d-81e4-6162afac0198-metrics-tls\") pod \"dns-default-v6ppc\" (UID: \"f921d091-7bf8-4d6d-81e4-6162afac0198\") " pod="openshift-dns/dns-default-v6ppc"
Sep 19 01:13:00.928684 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:00.927942    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 01:13:00.928684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927947    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bgf4\" (UniqueName: \"kubernetes.io/projected/f921d091-7bf8-4d6d-81e4-6162afac0198-kube-api-access-2bgf4\") pod \"dns-default-v6ppc\" (UID: \"f921d091-7bf8-4d6d-81e4-6162afac0198\") " pod="openshift-dns/dns-default-v6ppc"
Sep 19 01:13:00.928684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.927987    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpbbj\" (UniqueName: \"kubernetes.io/projected/8bdd86a3-9335-4e27-b183-c07df2869fb5-kube-api-access-fpbbj\") pod \"service-ca-operator-57cdc4b768-44vdn\" (UID: \"8bdd86a3-9335-4e27-b183-c07df2869fb5\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-44vdn"
Sep 19 01:13:00.928684 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:00.928001    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f921d091-7bf8-4d6d-81e4-6162afac0198-metrics-tls podName:f921d091-7bf8-4d6d-81e4-6162afac0198 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:01.427983052 +0000 UTC m=+36.334845091 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f921d091-7bf8-4d6d-81e4-6162afac0198-metrics-tls") pod "dns-default-v6ppc" (UID: "f921d091-7bf8-4d6d-81e4-6162afac0198") : secret "dns-default-metrics-tls" not found
Sep 19 01:13:00.928684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.928032    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/af99297d-b9c3-4e8d-b018-cb271f970733-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-f5gtc\" (UID: \"af99297d-b9c3-4e8d-b018-cb271f970733\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-f5gtc"
Sep 19 01:13:00.928684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.928054    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz7qc\" (UniqueName: \"kubernetes.io/projected/39355481-6d16-4553-961a-f03ca171e712-kube-api-access-jz7qc\") pod \"insights-operator-6f9c644dd9-p6l4m\" (UID: \"39355481-6d16-4553-961a-f03ca171e712\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m"
Sep 19 01:13:00.928684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.928073    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fg64\" (UniqueName: \"kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-kube-api-access-6fg64\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:00.928684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.928666    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39355481-6d16-4553-961a-f03ca171e712-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-p6l4m\" (UID: \"39355481-6d16-4553-961a-f03ca171e712\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m"
Sep 19 01:13:00.928684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.928670    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f921d091-7bf8-4d6d-81e4-6162afac0198-config-volume\") pod \"dns-default-v6ppc\" (UID: \"f921d091-7bf8-4d6d-81e4-6162afac0198\") " pod="openshift-dns/dns-default-v6ppc"
Sep 19 01:13:00.929203 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.928800    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39355481-6d16-4553-961a-f03ca171e712-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-p6l4m\" (UID: \"39355481-6d16-4553-961a-f03ca171e712\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m"
Sep 19 01:13:00.929256 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.929212    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a439b5fa-a142-4815-b145-c3278eea25a0-ca-trust-extracted\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:00.929594 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.929554    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a439b5fa-a142-4815-b145-c3278eea25a0-trusted-ca\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:00.932237 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.932215    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/39355481-6d16-4553-961a-f03ca171e712-serving-cert\") pod \"insights-operator-6f9c644dd9-p6l4m\" (UID: \"39355481-6d16-4553-961a-f03ca171e712\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m"
Sep 19 01:13:00.932480 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.932457    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a439b5fa-a142-4815-b145-c3278eea25a0-image-registry-private-configuration\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:00.932560 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.932537    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77q5s\" (UniqueName: \"kubernetes.io/projected/538e796f-8768-4bf9-ab1c-c3631bb14317-kube-api-access-77q5s\") pod \"network-check-target-nh6vq\" (UID: \"538e796f-8768-4bf9-ab1c-c3631bb14317\") " pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:13:00.932615 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.932591    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a439b5fa-a142-4815-b145-c3278eea25a0-installation-pull-secrets\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:00.942123 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.942101    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz7qc\" (UniqueName: \"kubernetes.io/projected/39355481-6d16-4553-961a-f03ca171e712-kube-api-access-jz7qc\") pod \"insights-operator-6f9c644dd9-p6l4m\" (UID: \"39355481-6d16-4553-961a-f03ca171e712\") " pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m"
Sep 19 01:13:00.942319 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.942275    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fg64\" (UniqueName: \"kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-kube-api-access-6fg64\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:00.942656 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.942622    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-bound-sa-token\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:00.943165 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.943146    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6t7t\" (UniqueName: \"kubernetes.io/projected/73e4b372-92e4-4eeb-ac37-6c5ec11217f0-kube-api-access-j6t7t\") pod \"volume-data-source-validator-5c4db8bdf9-zmkjg\" (UID: \"73e4b372-92e4-4eeb-ac37-6c5ec11217f0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-zmkjg"
Sep 19 01:13:00.943420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:00.943404    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bgf4\" (UniqueName: \"kubernetes.io/projected/f921d091-7bf8-4d6d-81e4-6162afac0198-kube-api-access-2bgf4\") pod \"dns-default-v6ppc\" (UID: \"f921d091-7bf8-4d6d-81e4-6162afac0198\") " pod="openshift-dns/dns-default-v6ppc"
Sep 19 01:13:01.028298 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.028227    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m"
Sep 19 01:13:01.028431 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.028380    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa9e1f35-1684-4704-8bd7-532245d3fc97-cert\") pod \"ingress-canary-p4cpb\" (UID: \"aa9e1f35-1684-4704-8bd7-532245d3fc97\") " pod="openshift-ingress-canary/ingress-canary-p4cpb"
Sep 19 01:13:01.028431 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.028419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpbbj\" (UniqueName: \"kubernetes.io/projected/8bdd86a3-9335-4e27-b183-c07df2869fb5-kube-api-access-fpbbj\") pod \"service-ca-operator-57cdc4b768-44vdn\" (UID: \"8bdd86a3-9335-4e27-b183-c07df2869fb5\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-44vdn"
Sep 19 01:13:01.028565 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.028452    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/af99297d-b9c3-4e8d-b018-cb271f970733-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-f5gtc\" (UID: \"af99297d-b9c3-4e8d-b018-cb271f970733\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-f5gtc"
Sep 19 01:13:01.028565 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.028511    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccmf6\" (UniqueName: \"kubernetes.io/projected/9cb55096-dad6-4d87-a63b-48e18dc7a011-kube-api-access-ccmf6\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:01.028565 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.028550    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/74c8e2f6-1b01-4bd1-b491-ad407a15a109-config\") pod \"console-operator-fd4bb46c-f6bdh\" (UID: \"74c8e2f6-1b01-4bd1-b491-ad407a15a109\") " pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh"
Sep 19 01:13:01.028565 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:01.028552    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 19 01:13:01.028750 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.028575    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/74c8e2f6-1b01-4bd1-b491-ad407a15a109-serving-cert\") pod \"console-operator-fd4bb46c-f6bdh\" (UID: \"74c8e2f6-1b01-4bd1-b491-ad407a15a109\") " pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh"
Sep 19 01:13:01.028750 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.028614    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsgkk\" (UniqueName: \"kubernetes.io/projected/2c41153e-4f57-4158-83b8-18563e2750e8-kube-api-access-fsgkk\") pod \"kube-storage-version-migrator-operator-55889f7b47-bvc7f\" (UID: \"2c41153e-4f57-4158-83b8-18563e2750e8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-bvc7f"
Sep 19 01:13:01.028750 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:01.028634    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af99297d-b9c3-4e8d-b018-cb271f970733-samples-operator-tls podName:af99297d-b9c3-4e8d-b018-cb271f970733 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:01.528615229 +0000 UTC m=+36.435477258 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/af99297d-b9c3-4e8d-b018-cb271f970733-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-f5gtc" (UID: "af99297d-b9c3-4e8d-b018-cb271f970733") : secret "samples-operator-tls" not found
Sep 19 01:13:01.028750 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.028664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c63ea3c5-742f-4077-a661-388d76e45d63-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-9bmsn\" (UID: \"c63ea3c5-742f-4077-a661-388d76e45d63\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9bmsn"
Sep 19 01:13:01.028750 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.028682    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2c41153e-4f57-4158-83b8-18563e2750e8-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-bvc7f\" (UID: \"2c41153e-4f57-4158-83b8-18563e2750e8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-bvc7f"
Sep 19 01:13:01.028750 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.028698    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-metrics-certs\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:01.028750 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.028719    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/74c8e2f6-1b01-4bd1-b491-ad407a15a109-trusted-ca\") pod \"console-operator-fd4bb46c-f6bdh\" (UID: \"74c8e2f6-1b01-4bd1-b491-ad407a15a109\") " pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh"
Sep 19 01:13:01.028750 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.028733    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-stats-auth\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:01.028750 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.028753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qht2p\" (UniqueName: \"kubernetes.io/projected/c63ea3c5-742f-4077-a661-388d76e45d63-kube-api-access-qht2p\") pod \"cluster-monitoring-operator-847b8bd8d7-9bmsn\" (UID: \"c63ea3c5-742f-4077-a661-388d76e45d63\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9bmsn"
Sep 19 01:13:01.029194 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:01.028786    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 01:13:01.029194 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:01.028886    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c63ea3c5-742f-4077-a661-388d76e45d63-cluster-monitoring-operator-tls podName:c63ea3c5-742f-4077-a661-388d76e45d63 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:01.528870737 +0000 UTC m=+36.435732762 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c63ea3c5-742f-4077-a661-388d76e45d63-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-9bmsn" (UID: "c63ea3c5-742f-4077-a661-388d76e45d63") : secret "cluster-monitoring-operator-tls" not found
Sep 19 01:13:01.029194 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.028786    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2c41153e-4f57-4158-83b8-18563e2750e8-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-bvc7f\" (UID: \"2c41153e-4f57-4158-83b8-18563e2750e8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-bvc7f"
Sep 19 01:13:01.029397 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.029369    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2c41153e-4f57-4158-83b8-18563e2750e8-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-bvc7f\" (UID: \"2c41153e-4f57-4158-83b8-18563e2750e8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-bvc7f"
Sep 19 01:13:01.029492 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.029469    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cb55096-dad6-4d87-a63b-48e18dc7a011-service-ca-bundle\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:01.029621 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.029540    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/74c8e2f6-1b01-4bd1-b491-ad407a15a109-config\") pod \"console-operator-fd4bb46c-f6bdh\" (UID: \"74c8e2f6-1b01-4bd1-b491-ad407a15a109\") " pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh"
Sep 19 01:13:01.029621 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.029573    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8bdd86a3-9335-4e27-b183-c07df2869fb5-serving-cert\") pod \"service-ca-operator-57cdc4b768-44vdn\" (UID: \"8bdd86a3-9335-4e27-b183-c07df2869fb5\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-44vdn"
Sep 19 01:13:01.029742 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.029624    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c63ea3c5-742f-4077-a661-388d76e45d63-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-9bmsn\" (UID: \"c63ea3c5-742f-4077-a661-388d76e45d63\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9bmsn"
Sep 19 01:13:01.029742 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.029657    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/74c8e2f6-1b01-4bd1-b491-ad407a15a109-trusted-ca\") pod \"console-operator-fd4bb46c-f6bdh\" (UID: \"74c8e2f6-1b01-4bd1-b491-ad407a15a109\") " pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh"
Sep 19 01:13:01.029742 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.029667    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tsbb\" (UniqueName: \"kubernetes.io/projected/aa9e1f35-1684-4704-8bd7-532245d3fc97-kube-api-access-5tsbb\") pod \"ingress-canary-p4cpb\" (UID: \"aa9e1f35-1684-4704-8bd7-532245d3fc97\") " pod="openshift-ingress-canary/ingress-canary-p4cpb"
Sep 19 01:13:01.029742 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.029711    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq67w\" (UniqueName: \"kubernetes.io/projected/74c8e2f6-1b01-4bd1-b491-ad407a15a109-kube-api-access-kq67w\") pod \"console-operator-fd4bb46c-f6bdh\" (UID: \"74c8e2f6-1b01-4bd1-b491-ad407a15a109\") " pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh"
Sep 19 01:13:01.029742 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.029737    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8bdd86a3-9335-4e27-b183-c07df2869fb5-config\") pod \"service-ca-operator-57cdc4b768-44vdn\" (UID: \"8bdd86a3-9335-4e27-b183-c07df2869fb5\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-44vdn"
Sep 19 01:13:01.029993 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.029762    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nrsf\" (UniqueName: \"kubernetes.io/projected/af99297d-b9c3-4e8d-b018-cb271f970733-kube-api-access-8nrsf\") pod \"cluster-samples-operator-685d7d68b5-f5gtc\" (UID: \"af99297d-b9c3-4e8d-b018-cb271f970733\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-f5gtc"
Sep 19 01:13:01.029993 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.029796    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kwfz\" (UniqueName: \"kubernetes.io/projected/4f03315a-a129-4ccd-870e-e57ae72c878a-kube-api-access-7kwfz\") pod \"network-check-source-7cbc45fddc-qxlvd\" (UID: \"4f03315a-a129-4ccd-870e-e57ae72c878a\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-qxlvd"
Sep 19 01:13:01.029993 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.029834    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-default-certificate\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:01.030324 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.030270    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8bdd86a3-9335-4e27-b183-c07df2869fb5-config\") pod \"service-ca-operator-57cdc4b768-44vdn\" (UID: \"8bdd86a3-9335-4e27-b183-c07df2869fb5\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-44vdn"
Sep 19 01:13:01.030324 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.030287    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c63ea3c5-742f-4077-a661-388d76e45d63-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-9bmsn\" (UID: \"c63ea3c5-742f-4077-a661-388d76e45d63\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9bmsn"
Sep 19 01:13:01.031320 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.031299    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/74c8e2f6-1b01-4bd1-b491-ad407a15a109-serving-cert\") pod \"console-operator-fd4bb46c-f6bdh\" (UID: \"74c8e2f6-1b01-4bd1-b491-ad407a15a109\") " pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh"
Sep 19 01:13:01.031684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.031666    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2c41153e-4f57-4158-83b8-18563e2750e8-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-bvc7f\" (UID: \"2c41153e-4f57-4158-83b8-18563e2750e8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-bvc7f"
Sep 19 01:13:01.032177 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.032147    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8bdd86a3-9335-4e27-b183-c07df2869fb5-serving-cert\") pod \"service-ca-operator-57cdc4b768-44vdn\" (UID: \"8bdd86a3-9335-4e27-b183-c07df2869fb5\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-44vdn"
Sep 19 01:13:01.041172 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.041156    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-zmkjg"
Sep 19 01:13:01.059841 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.059781    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsgkk\" (UniqueName: \"kubernetes.io/projected/2c41153e-4f57-4158-83b8-18563e2750e8-kube-api-access-fsgkk\") pod \"kube-storage-version-migrator-operator-55889f7b47-bvc7f\" (UID: \"2c41153e-4f57-4158-83b8-18563e2750e8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-bvc7f"
Sep 19 01:13:01.060663 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.060412    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qht2p\" (UniqueName: \"kubernetes.io/projected/c63ea3c5-742f-4077-a661-388d76e45d63-kube-api-access-qht2p\") pod \"cluster-monitoring-operator-847b8bd8d7-9bmsn\" (UID: \"c63ea3c5-742f-4077-a661-388d76e45d63\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9bmsn"
Sep 19 01:13:01.060663 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.060472    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpbbj\" (UniqueName: \"kubernetes.io/projected/8bdd86a3-9335-4e27-b183-c07df2869fb5-kube-api-access-fpbbj\") pod \"service-ca-operator-57cdc4b768-44vdn\" (UID: \"8bdd86a3-9335-4e27-b183-c07df2869fb5\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-44vdn"
Sep 19 01:13:01.060663 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.060602    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nrsf\" (UniqueName: \"kubernetes.io/projected/af99297d-b9c3-4e8d-b018-cb271f970733-kube-api-access-8nrsf\") pod \"cluster-samples-operator-685d7d68b5-f5gtc\" (UID: \"af99297d-b9c3-4e8d-b018-cb271f970733\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-f5gtc"
Sep 19 01:13:01.060663 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.060611    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq67w\" (UniqueName: \"kubernetes.io/projected/74c8e2f6-1b01-4bd1-b491-ad407a15a109-kube-api-access-kq67w\") pod \"console-operator-fd4bb46c-f6bdh\" (UID: \"74c8e2f6-1b01-4bd1-b491-ad407a15a109\") " pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh"
Sep 19 01:13:01.062140 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.062126    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:13:01.065899 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.065882    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f5jnh\""
Sep 19 01:13:01.071451 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.071436    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:13:01.089110 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.089088    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-44vdn"
Sep 19 01:13:01.105001 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.104983    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh"
Sep 19 01:13:01.130110 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.130091    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tsbb\" (UniqueName: \"kubernetes.io/projected/aa9e1f35-1684-4704-8bd7-532245d3fc97-kube-api-access-5tsbb\") pod \"ingress-canary-p4cpb\" (UID: \"aa9e1f35-1684-4704-8bd7-532245d3fc97\") " pod="openshift-ingress-canary/ingress-canary-p4cpb"
Sep 19 01:13:01.130232 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.130140    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kwfz\" (UniqueName: \"kubernetes.io/projected/4f03315a-a129-4ccd-870e-e57ae72c878a-kube-api-access-7kwfz\") pod \"network-check-source-7cbc45fddc-qxlvd\" (UID: \"4f03315a-a129-4ccd-870e-e57ae72c878a\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-qxlvd"
Sep 19 01:13:01.130232 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.130186    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-default-certificate\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:01.130232 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.130209    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa9e1f35-1684-4704-8bd7-532245d3fc97-cert\") pod \"ingress-canary-p4cpb\" (UID: \"aa9e1f35-1684-4704-8bd7-532245d3fc97\") " pod="openshift-ingress-canary/ingress-canary-p4cpb"
Sep 19 01:13:01.130386 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.130355    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccmf6\" (UniqueName: \"kubernetes.io/projected/9cb55096-dad6-4d87-a63b-48e18dc7a011-kube-api-access-ccmf6\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:01.130465 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.130445    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-metrics-certs\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:01.130543 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.130491    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-stats-auth\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:01.130608 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.130572    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cb55096-dad6-4d87-a63b-48e18dc7a011-service-ca-bundle\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:01.130659 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:01.130618    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 19 01:13:01.130706 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:01.130681    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-metrics-certs podName:9cb55096-dad6-4d87-a63b-48e18dc7a011 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:01.630660904 +0000 UTC m=+36.537522931 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-metrics-certs") pod "router-default-69d68c9d66-2gblx" (UID: "9cb55096-dad6-4d87-a63b-48e18dc7a011") : secret "router-metrics-certs-default" not found
Sep 19 01:13:01.130762 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:01.130742    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9cb55096-dad6-4d87-a63b-48e18dc7a011-service-ca-bundle podName:9cb55096-dad6-4d87-a63b-48e18dc7a011 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:01.630725766 +0000 UTC m=+36.537587817 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9cb55096-dad6-4d87-a63b-48e18dc7a011-service-ca-bundle") pod "router-default-69d68c9d66-2gblx" (UID: "9cb55096-dad6-4d87-a63b-48e18dc7a011") : configmap references non-existent config key: service-ca.crt
Sep 19 01:13:01.132818 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.132800    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa9e1f35-1684-4704-8bd7-532245d3fc97-cert\") pod \"ingress-canary-p4cpb\" (UID: \"aa9e1f35-1684-4704-8bd7-532245d3fc97\") " pod="openshift-ingress-canary/ingress-canary-p4cpb"
Sep 19 01:13:01.132954 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.132938    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-default-certificate\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:01.133014 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.132992    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-stats-auth\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:01.148355 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.148325    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-bvc7f"
Sep 19 01:13:01.150741 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.150721    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tsbb\" (UniqueName: \"kubernetes.io/projected/aa9e1f35-1684-4704-8bd7-532245d3fc97-kube-api-access-5tsbb\") pod \"ingress-canary-p4cpb\" (UID: \"aa9e1f35-1684-4704-8bd7-532245d3fc97\") " pod="openshift-ingress-canary/ingress-canary-p4cpb"
Sep 19 01:13:01.151060 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.151036    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kwfz\" (UniqueName: \"kubernetes.io/projected/4f03315a-a129-4ccd-870e-e57ae72c878a-kube-api-access-7kwfz\") pod \"network-check-source-7cbc45fddc-qxlvd\" (UID: \"4f03315a-a129-4ccd-870e-e57ae72c878a\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-qxlvd"
Sep 19 01:13:01.151414 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.151398    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccmf6\" (UniqueName: \"kubernetes.io/projected/9cb55096-dad6-4d87-a63b-48e18dc7a011-kube-api-access-ccmf6\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:01.186990 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.186969    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p4cpb"
Sep 19 01:13:01.200127 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.200104    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-qxlvd"
Sep 19 01:13:01.432608 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.432532    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f921d091-7bf8-4d6d-81e4-6162afac0198-metrics-tls\") pod \"dns-default-v6ppc\" (UID: \"f921d091-7bf8-4d6d-81e4-6162afac0198\") " pod="openshift-dns/dns-default-v6ppc"
Sep 19 01:13:01.432759 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.432653    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-registry-tls\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:01.432759 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:01.432705    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 01:13:01.432856 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:01.432762    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 01:13:01.432856 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:01.432776    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw: secret "image-registry-tls" not found
Sep 19 01:13:01.432856 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:01.432782    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f921d091-7bf8-4d6d-81e4-6162afac0198-metrics-tls podName:f921d091-7bf8-4d6d-81e4-6162afac0198 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:02.432762881 +0000 UTC m=+37.339624911 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f921d091-7bf8-4d6d-81e4-6162afac0198-metrics-tls") pod "dns-default-v6ppc" (UID: "f921d091-7bf8-4d6d-81e4-6162afac0198") : secret "dns-default-metrics-tls" not found
Sep 19 01:13:01.432856 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:01.432817    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-registry-tls podName:a439b5fa-a142-4815-b145-c3278eea25a0 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:02.432802772 +0000 UTC m=+37.339664814 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-registry-tls") pod "image-registry-5d7bd8c8fd-kxlmw" (UID: "a439b5fa-a142-4815-b145-c3278eea25a0") : secret "image-registry-tls" not found
Sep 19 01:13:01.533945 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.533904    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/af99297d-b9c3-4e8d-b018-cb271f970733-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-f5gtc\" (UID: \"af99297d-b9c3-4e8d-b018-cb271f970733\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-f5gtc"
Sep 19 01:13:01.534114 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.533955    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c63ea3c5-742f-4077-a661-388d76e45d63-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-9bmsn\" (UID: \"c63ea3c5-742f-4077-a661-388d76e45d63\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9bmsn"
Sep 19 01:13:01.534114 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:01.534068    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 19 01:13:01.534114 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:01.534080    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 01:13:01.534291 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:01.534137    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af99297d-b9c3-4e8d-b018-cb271f970733-samples-operator-tls podName:af99297d-b9c3-4e8d-b018-cb271f970733 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:02.534120482 +0000 UTC m=+37.440982526 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/af99297d-b9c3-4e8d-b018-cb271f970733-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-f5gtc" (UID: "af99297d-b9c3-4e8d-b018-cb271f970733") : secret "samples-operator-tls" not found
Sep 19 01:13:01.534291 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:01.534158    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c63ea3c5-742f-4077-a661-388d76e45d63-cluster-monitoring-operator-tls podName:c63ea3c5-742f-4077-a661-388d76e45d63 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:02.534148467 +0000 UTC m=+37.441010509 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c63ea3c5-742f-4077-a661-388d76e45d63-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-9bmsn" (UID: "c63ea3c5-742f-4077-a661-388d76e45d63") : secret "cluster-monitoring-operator-tls" not found
Sep 19 01:13:01.634733 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.634687    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-metrics-certs\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:01.634882 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:01.634760    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cb55096-dad6-4d87-a63b-48e18dc7a011-service-ca-bundle\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:01.634882 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:01.634843    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 19 01:13:01.635005 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:01.634917    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9cb55096-dad6-4d87-a63b-48e18dc7a011-service-ca-bundle podName:9cb55096-dad6-4d87-a63b-48e18dc7a011 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:02.634897136 +0000 UTC m=+37.541759177 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9cb55096-dad6-4d87-a63b-48e18dc7a011-service-ca-bundle") pod "router-default-69d68c9d66-2gblx" (UID: "9cb55096-dad6-4d87-a63b-48e18dc7a011") : configmap references non-existent config key: service-ca.crt
Sep 19 01:13:01.635005 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:01.634941    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-metrics-certs podName:9cb55096-dad6-4d87-a63b-48e18dc7a011 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:02.634934827 +0000 UTC m=+37.541796858 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-metrics-certs") pod "router-default-69d68c9d66-2gblx" (UID: "9cb55096-dad6-4d87-a63b-48e18dc7a011") : secret "router-metrics-certs-default" not found
Sep 19 01:13:02.057077 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.056867    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:13:02.057224 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.056923    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:13:02.059507 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.059478    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 19 01:13:02.060045 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.060032    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 19 01:13:02.060453 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.060135    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lthdr\""
Sep 19 01:13:02.180672 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.180632    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-qxlvd"]
Sep 19 01:13:02.182523 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.182485    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-zmkjg"]
Sep 19 01:13:02.183446 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.183421    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-44vdn"]
Sep 19 01:13:02.183712 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:13:02.183641    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f03315a_a129_4ccd_870e_e57ae72c878a.slice/crio-5b4d283fac4a71cbb5ab58921ee2cbc524c9a0717fbc8c52317f751e4ef6af1a WatchSource:0}: Error finding container 5b4d283fac4a71cbb5ab58921ee2cbc524c9a0717fbc8c52317f751e4ef6af1a: Status 404 returned error can't find the container with id 5b4d283fac4a71cbb5ab58921ee2cbc524c9a0717fbc8c52317f751e4ef6af1a
Sep 19 01:13:02.184279 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:13:02.184255    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73e4b372_92e4_4eeb_ac37_6c5ec11217f0.slice/crio-1e8afe6e0a74b093f9595aabc39d54a230218ac55c5bbb16e72116456866ed11 WatchSource:0}: Error finding container 1e8afe6e0a74b093f9595aabc39d54a230218ac55c5bbb16e72116456866ed11: Status 404 returned error can't find the container with id 1e8afe6e0a74b093f9595aabc39d54a230218ac55c5bbb16e72116456866ed11
Sep 19 01:13:02.185078 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:13:02.185042    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bdd86a3_9335_4e27_b183_c07df2869fb5.slice/crio-8e1ab4a33e6885a373e1241aad8061809d557055e1655a7e7610fb49ed64bf0b WatchSource:0}: Error finding container 8e1ab4a33e6885a373e1241aad8061809d557055e1655a7e7610fb49ed64bf0b: Status 404 returned error can't find the container with id 8e1ab4a33e6885a373e1241aad8061809d557055e1655a7e7610fb49ed64bf0b
Sep 19 01:13:02.190901 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.190742    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-f6bdh"]
Sep 19 01:13:02.192316 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.192177    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nh6vq"]
Sep 19 01:13:02.193075 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.193032    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p4cpb"]
Sep 19 01:13:02.197578 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:13:02.197555    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74c8e2f6_1b01_4bd1_b491_ad407a15a109.slice/crio-289c09c7044d8851b9086ecac27eed54114428493358b9e22f472be502d289dd WatchSource:0}: Error finding container 289c09c7044d8851b9086ecac27eed54114428493358b9e22f472be502d289dd: Status 404 returned error can't find the container with id 289c09c7044d8851b9086ecac27eed54114428493358b9e22f472be502d289dd
Sep 19 01:13:02.200419 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.199286    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-p6l4m"]
Sep 19 01:13:02.202956 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.202712    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-bvc7f"]
Sep 19 01:13:02.207756 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:13:02.207730    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c41153e_4f57_4158_83b8_18563e2750e8.slice/crio-e40627c3940a8c67fd0d16ff4df4c7a917724e3323f04d42e5ec19946abe59cd WatchSource:0}: Error finding container e40627c3940a8c67fd0d16ff4df4c7a917724e3323f04d42e5ec19946abe59cd: Status 404 returned error can't find the container with id e40627c3940a8c67fd0d16ff4df4c7a917724e3323f04d42e5ec19946abe59cd
Sep 19 01:13:02.300828 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.300796    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nh6vq" event={"ID":"538e796f-8768-4bf9-ab1c-c3631bb14317","Type":"ContainerStarted","Data":"f1dcf81d4148405bd21bd20b673ce8548ed72aeee172b0873f8283183c7718fb"}
Sep 19 01:13:02.302163 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.302136    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-qxlvd" event={"ID":"4f03315a-a129-4ccd-870e-e57ae72c878a","Type":"ContainerStarted","Data":"5b4d283fac4a71cbb5ab58921ee2cbc524c9a0717fbc8c52317f751e4ef6af1a"}
Sep 19 01:13:02.303170 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.303147    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p4cpb" event={"ID":"aa9e1f35-1684-4704-8bd7-532245d3fc97","Type":"ContainerStarted","Data":"24198c7ff69b8542c8f65c2317b5572250be59b280a4a26da837ede94ab89423"}
Sep 19 01:13:02.304430 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.304405    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-zmkjg" event={"ID":"73e4b372-92e4-4eeb-ac37-6c5ec11217f0","Type":"ContainerStarted","Data":"1e8afe6e0a74b093f9595aabc39d54a230218ac55c5bbb16e72116456866ed11"}
Sep 19 01:13:02.305390 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.305373    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-bvc7f" event={"ID":"2c41153e-4f57-4158-83b8-18563e2750e8","Type":"ContainerStarted","Data":"e40627c3940a8c67fd0d16ff4df4c7a917724e3323f04d42e5ec19946abe59cd"}
Sep 19 01:13:02.306316 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.306299    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-44vdn" event={"ID":"8bdd86a3-9335-4e27-b183-c07df2869fb5","Type":"ContainerStarted","Data":"8e1ab4a33e6885a373e1241aad8061809d557055e1655a7e7610fb49ed64bf0b"}
Sep 19 01:13:02.307307 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.307259    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh" event={"ID":"74c8e2f6-1b01-4bd1-b491-ad407a15a109","Type":"ContainerStarted","Data":"289c09c7044d8851b9086ecac27eed54114428493358b9e22f472be502d289dd"}
Sep 19 01:13:02.308174 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.308144    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m" event={"ID":"39355481-6d16-4553-961a-f03ca171e712","Type":"ContainerStarted","Data":"fb63e39e965c15c7fab7ed152294dda5ac937f2373845220f46d88ea683af3c3"}
Sep 19 01:13:02.442315 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.442281    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-registry-tls\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:02.442448 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:02.442427    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 01:13:02.442484 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:02.442451    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw: secret "image-registry-tls" not found
Sep 19 01:13:02.442565 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.442432    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f921d091-7bf8-4d6d-81e4-6162afac0198-metrics-tls\") pod \"dns-default-v6ppc\" (UID: \"f921d091-7bf8-4d6d-81e4-6162afac0198\") " pod="openshift-dns/dns-default-v6ppc"
Sep 19 01:13:02.442565 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:02.442527    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-registry-tls podName:a439b5fa-a142-4815-b145-c3278eea25a0 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:04.442492376 +0000 UTC m=+39.349354403 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-registry-tls") pod "image-registry-5d7bd8c8fd-kxlmw" (UID: "a439b5fa-a142-4815-b145-c3278eea25a0") : secret "image-registry-tls" not found
Sep 19 01:13:02.442649 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:02.442542    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 01:13:02.442649 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:02.442633    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f921d091-7bf8-4d6d-81e4-6162afac0198-metrics-tls podName:f921d091-7bf8-4d6d-81e4-6162afac0198 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:04.442619008 +0000 UTC m=+39.349481033 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f921d091-7bf8-4d6d-81e4-6162afac0198-metrics-tls") pod "dns-default-v6ppc" (UID: "f921d091-7bf8-4d6d-81e4-6162afac0198") : secret "dns-default-metrics-tls" not found
Sep 19 01:13:02.543255 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.543225    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/af99297d-b9c3-4e8d-b018-cb271f970733-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-f5gtc\" (UID: \"af99297d-b9c3-4e8d-b018-cb271f970733\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-f5gtc"
Sep 19 01:13:02.543396 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.543271    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c63ea3c5-742f-4077-a661-388d76e45d63-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-9bmsn\" (UID: \"c63ea3c5-742f-4077-a661-388d76e45d63\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9bmsn"
Sep 19 01:13:02.543396 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:02.543387    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 01:13:02.543516 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:02.543391    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 19 01:13:02.543516 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:02.543446    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c63ea3c5-742f-4077-a661-388d76e45d63-cluster-monitoring-operator-tls podName:c63ea3c5-742f-4077-a661-388d76e45d63 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:04.543432671 +0000 UTC m=+39.450294701 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c63ea3c5-742f-4077-a661-388d76e45d63-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-9bmsn" (UID: "c63ea3c5-742f-4077-a661-388d76e45d63") : secret "cluster-monitoring-operator-tls" not found
Sep 19 01:13:02.543516 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:02.543458    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af99297d-b9c3-4e8d-b018-cb271f970733-samples-operator-tls podName:af99297d-b9c3-4e8d-b018-cb271f970733 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:04.543452636 +0000 UTC m=+39.450314661 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/af99297d-b9c3-4e8d-b018-cb271f970733-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-f5gtc" (UID: "af99297d-b9c3-4e8d-b018-cb271f970733") : secret "samples-operator-tls" not found
Sep 19 01:13:02.644304 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.644222    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cb55096-dad6-4d87-a63b-48e18dc7a011-service-ca-bundle\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:02.644456 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:02.644362    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-metrics-certs\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:02.644456 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:02.644398    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9cb55096-dad6-4d87-a63b-48e18dc7a011-service-ca-bundle podName:9cb55096-dad6-4d87-a63b-48e18dc7a011 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:04.644380026 +0000 UTC m=+39.551242056 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9cb55096-dad6-4d87-a63b-48e18dc7a011-service-ca-bundle") pod "router-default-69d68c9d66-2gblx" (UID: "9cb55096-dad6-4d87-a63b-48e18dc7a011") : configmap references non-existent config key: service-ca.crt
Sep 19 01:13:02.644601 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:02.644456    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 19 01:13:02.644601 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:02.644531    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-metrics-certs podName:9cb55096-dad6-4d87-a63b-48e18dc7a011 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:04.644514576 +0000 UTC m=+39.551376620 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-metrics-certs") pod "router-default-69d68c9d66-2gblx" (UID: "9cb55096-dad6-4d87-a63b-48e18dc7a011") : secret "router-metrics-certs-default" not found
Sep 19 01:13:03.323308 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:03.323270    2580 generic.go:356] "Generic (PLEG): container finished" podID="23fcc30f-348f-4c04-aff2-23fe3420219a" containerID="781b57af9c198c6d7fad12b504477ece3b38a1af0fd62539131254c2f663437c" exitCode=0
Sep 19 01:13:03.324040 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:03.323322    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tfq72" event={"ID":"23fcc30f-348f-4c04-aff2-23fe3420219a","Type":"ContainerDied","Data":"781b57af9c198c6d7fad12b504477ece3b38a1af0fd62539131254c2f663437c"}
Sep 19 01:13:04.345418 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:04.345327    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tfq72" event={"ID":"23fcc30f-348f-4c04-aff2-23fe3420219a","Type":"ContainerStarted","Data":"fc798fbe7bb97622c3f03a0bebe2ab9c8d05f18e3de35e89129565b74ec59d5b"}
Sep 19 01:13:04.462453 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:04.462087    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-registry-tls\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:04.462453 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:04.462245    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f921d091-7bf8-4d6d-81e4-6162afac0198-metrics-tls\") pod \"dns-default-v6ppc\" (UID: \"f921d091-7bf8-4d6d-81e4-6162afac0198\") " pod="openshift-dns/dns-default-v6ppc"
Sep 19 01:13:04.462453 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:04.462378    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 01:13:04.462453 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:04.462437    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f921d091-7bf8-4d6d-81e4-6162afac0198-metrics-tls podName:f921d091-7bf8-4d6d-81e4-6162afac0198 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:08.462419147 +0000 UTC m=+43.369281184 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f921d091-7bf8-4d6d-81e4-6162afac0198-metrics-tls") pod "dns-default-v6ppc" (UID: "f921d091-7bf8-4d6d-81e4-6162afac0198") : secret "dns-default-metrics-tls" not found
Sep 19 01:13:04.463378 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:04.463355    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 01:13:04.463378 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:04.463380    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw: secret "image-registry-tls" not found
Sep 19 01:13:04.463543 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:04.463424    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-registry-tls podName:a439b5fa-a142-4815-b145-c3278eea25a0 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:08.463408604 +0000 UTC m=+43.370270633 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-registry-tls") pod "image-registry-5d7bd8c8fd-kxlmw" (UID: "a439b5fa-a142-4815-b145-c3278eea25a0") : secret "image-registry-tls" not found
Sep 19 01:13:04.563713 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:04.563669    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/af99297d-b9c3-4e8d-b018-cb271f970733-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-f5gtc\" (UID: \"af99297d-b9c3-4e8d-b018-cb271f970733\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-f5gtc"
Sep 19 01:13:04.563860 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:04.563739    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c63ea3c5-742f-4077-a661-388d76e45d63-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-9bmsn\" (UID: \"c63ea3c5-742f-4077-a661-388d76e45d63\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9bmsn"
Sep 19 01:13:04.563997 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:04.563981    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 01:13:04.564061 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:04.564051    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c63ea3c5-742f-4077-a661-388d76e45d63-cluster-monitoring-operator-tls podName:c63ea3c5-742f-4077-a661-388d76e45d63 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:08.564031416 +0000 UTC m=+43.470893445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c63ea3c5-742f-4077-a661-388d76e45d63-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-9bmsn" (UID: "c63ea3c5-742f-4077-a661-388d76e45d63") : secret "cluster-monitoring-operator-tls" not found
Sep 19 01:13:04.564492 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:04.564472    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 19 01:13:04.564598 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:04.564542    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af99297d-b9c3-4e8d-b018-cb271f970733-samples-operator-tls podName:af99297d-b9c3-4e8d-b018-cb271f970733 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:08.564526741 +0000 UTC m=+43.471388784 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/af99297d-b9c3-4e8d-b018-cb271f970733-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-f5gtc" (UID: "af99297d-b9c3-4e8d-b018-cb271f970733") : secret "samples-operator-tls" not found
Sep 19 01:13:04.665386 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:04.665285    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-metrics-certs\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:04.665386 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:04.665382    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cb55096-dad6-4d87-a63b-48e18dc7a011-service-ca-bundle\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:04.665643 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:04.665607    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9cb55096-dad6-4d87-a63b-48e18dc7a011-service-ca-bundle podName:9cb55096-dad6-4d87-a63b-48e18dc7a011 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:08.66557781 +0000 UTC m=+43.572439860 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9cb55096-dad6-4d87-a63b-48e18dc7a011-service-ca-bundle") pod "router-default-69d68c9d66-2gblx" (UID: "9cb55096-dad6-4d87-a63b-48e18dc7a011") : configmap references non-existent config key: service-ca.crt
Sep 19 01:13:04.665823 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:04.665770    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 19 01:13:04.665823 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:04.665821    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-metrics-certs podName:9cb55096-dad6-4d87-a63b-48e18dc7a011 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:08.665807249 +0000 UTC m=+43.572669277 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-metrics-certs") pod "router-default-69d68c9d66-2gblx" (UID: "9cb55096-dad6-4d87-a63b-48e18dc7a011") : secret "router-metrics-certs-default" not found
Sep 19 01:13:05.357524 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:05.356954    2580 generic.go:356] "Generic (PLEG): container finished" podID="23fcc30f-348f-4c04-aff2-23fe3420219a" containerID="fc798fbe7bb97622c3f03a0bebe2ab9c8d05f18e3de35e89129565b74ec59d5b" exitCode=0
Sep 19 01:13:05.357524 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:05.357010    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tfq72" event={"ID":"23fcc30f-348f-4c04-aff2-23fe3420219a","Type":"ContainerDied","Data":"fc798fbe7bb97622c3f03a0bebe2ab9c8d05f18e3de35e89129565b74ec59d5b"}
Sep 19 01:13:08.507133 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:08.506888    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f921d091-7bf8-4d6d-81e4-6162afac0198-metrics-tls\") pod \"dns-default-v6ppc\" (UID: \"f921d091-7bf8-4d6d-81e4-6162afac0198\") " pod="openshift-dns/dns-default-v6ppc"
Sep 19 01:13:08.507600 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:08.507035    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 01:13:08.507600 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:08.507236    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-registry-tls\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:08.507600 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:08.507262    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f921d091-7bf8-4d6d-81e4-6162afac0198-metrics-tls podName:f921d091-7bf8-4d6d-81e4-6162afac0198 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:16.507230379 +0000 UTC m=+51.414092409 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f921d091-7bf8-4d6d-81e4-6162afac0198-metrics-tls") pod "dns-default-v6ppc" (UID: "f921d091-7bf8-4d6d-81e4-6162afac0198") : secret "dns-default-metrics-tls" not found
Sep 19 01:13:08.507600 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:08.507323    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 01:13:08.507600 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:08.507337    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw: secret "image-registry-tls" not found
Sep 19 01:13:08.507600 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:08.507382    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-registry-tls podName:a439b5fa-a142-4815-b145-c3278eea25a0 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:16.507368306 +0000 UTC m=+51.414230337 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-registry-tls") pod "image-registry-5d7bd8c8fd-kxlmw" (UID: "a439b5fa-a142-4815-b145-c3278eea25a0") : secret "image-registry-tls" not found
Sep 19 01:13:08.608273 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:08.608235    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c63ea3c5-742f-4077-a661-388d76e45d63-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-9bmsn\" (UID: \"c63ea3c5-742f-4077-a661-388d76e45d63\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9bmsn"
Sep 19 01:13:08.608440 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:08.608355    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 01:13:08.608440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:08.608383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/af99297d-b9c3-4e8d-b018-cb271f970733-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-f5gtc\" (UID: \"af99297d-b9c3-4e8d-b018-cb271f970733\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-f5gtc"
Sep 19 01:13:08.608440 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:08.608432    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c63ea3c5-742f-4077-a661-388d76e45d63-cluster-monitoring-operator-tls podName:c63ea3c5-742f-4077-a661-388d76e45d63 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:16.608411744 +0000 UTC m=+51.515273791 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c63ea3c5-742f-4077-a661-388d76e45d63-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-9bmsn" (UID: "c63ea3c5-742f-4077-a661-388d76e45d63") : secret "cluster-monitoring-operator-tls" not found
Sep 19 01:13:08.608643 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:08.608481    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 19 01:13:08.608643 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:08.608541    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af99297d-b9c3-4e8d-b018-cb271f970733-samples-operator-tls podName:af99297d-b9c3-4e8d-b018-cb271f970733 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:16.608528459 +0000 UTC m=+51.515390488 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/af99297d-b9c3-4e8d-b018-cb271f970733-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-f5gtc" (UID: "af99297d-b9c3-4e8d-b018-cb271f970733") : secret "samples-operator-tls" not found
Sep 19 01:13:08.709027 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:08.708987    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-metrics-certs\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:08.709200 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:08.709059    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cb55096-dad6-4d87-a63b-48e18dc7a011-service-ca-bundle\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:08.709200 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:08.709177    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 19 01:13:08.709321 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:08.709244    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-metrics-certs podName:9cb55096-dad6-4d87-a63b-48e18dc7a011 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:16.709227668 +0000 UTC m=+51.616089695 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-metrics-certs") pod "router-default-69d68c9d66-2gblx" (UID: "9cb55096-dad6-4d87-a63b-48e18dc7a011") : secret "router-metrics-certs-default" not found
Sep 19 01:13:08.709321 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:08.709259    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9cb55096-dad6-4d87-a63b-48e18dc7a011-service-ca-bundle podName:9cb55096-dad6-4d87-a63b-48e18dc7a011 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:16.709253367 +0000 UTC m=+51.616115393 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9cb55096-dad6-4d87-a63b-48e18dc7a011-service-ca-bundle") pod "router-default-69d68c9d66-2gblx" (UID: "9cb55096-dad6-4d87-a63b-48e18dc7a011") : configmap references non-existent config key: service-ca.crt
Sep 19 01:13:10.372642 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:10.372566    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tfq72" event={"ID":"23fcc30f-348f-4c04-aff2-23fe3420219a","Type":"ContainerStarted","Data":"a38847ae4bbda86bad4e263c2252d83658c8b049bedcef83a61f161566e95eb9"}
Sep 19 01:13:10.374088 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:10.374065    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-zmkjg" event={"ID":"73e4b372-92e4-4eeb-ac37-6c5ec11217f0","Type":"ContainerStarted","Data":"f6842723e2c427be996ce814526cc29149bf391bb3a7e4efcc41598beadfb0a3"}
Sep 19 01:13:10.394583 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:10.394546    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tfq72" podStartSLOduration=14.330696259 podStartE2EDuration="43.39453333s" totalImagesPullingTime="29.063837071s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 01:12:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:13:10.393672328 +0000 UTC m=+45.300534375" watchObservedRunningTime="2026-09-19 01:13:10.39453333 +0000 UTC m=+45.301395375"
Sep 19 01:13:10.409744 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:10.409706    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-zmkjg" podStartSLOduration=7.460481038 podStartE2EDuration="15.409692087s" totalImagesPullingTime="7.949211049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:12:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:13:10.408953619 +0000 UTC m=+45.315815666" watchObservedRunningTime="2026-09-19 01:13:10.409692087 +0000 UTC m=+45.316554134"
Sep 19 01:13:11.380597 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:11.380348    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-qxlvd" event={"ID":"4f03315a-a129-4ccd-870e-e57ae72c878a","Type":"ContainerStarted","Data":"be1ceb55ef8c342520b3ab486ea20a78d6c1b2620c40536992775953d83a6c84"}
Sep 19 01:13:11.382198 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:11.382162    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p4cpb" event={"ID":"aa9e1f35-1684-4704-8bd7-532245d3fc97","Type":"ContainerStarted","Data":"a7e96f7c34de3c5b06ec66c6c8770d78cbdd7b815bd04dc62d791783388e2251"}
Sep 19 01:13:11.383825 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:11.383775    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-bvc7f" event={"ID":"2c41153e-4f57-4158-83b8-18563e2750e8","Type":"ContainerStarted","Data":"009a7a7acb73749cf1e2629a1021dfd162fc64382794e468553fcb93a5980f55"}
Sep 19 01:13:11.385631 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:11.385605    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-44vdn" event={"ID":"8bdd86a3-9335-4e27-b183-c07df2869fb5","Type":"ContainerStarted","Data":"a2c0bc83f0a9ceb0662a5c08db5e3f38cae5e87ec696a4008ffca88254caec08"}
Sep 19 01:13:11.387354 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:11.387336    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f6bdh_74c8e2f6-1b01-4bd1-b491-ad407a15a109/console-operator/0.log"
Sep 19 01:13:11.387434 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:11.387373    2580 generic.go:356] "Generic (PLEG): container finished" podID="74c8e2f6-1b01-4bd1-b491-ad407a15a109" containerID="518b454335eec3fe03d385f04954782ad6ed130f47baf01e8255655783f970ec" exitCode=255
Sep 19 01:13:11.387489 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:11.387459    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh" event={"ID":"74c8e2f6-1b01-4bd1-b491-ad407a15a109","Type":"ContainerDied","Data":"518b454335eec3fe03d385f04954782ad6ed130f47baf01e8255655783f970ec"}
Sep 19 01:13:11.387648 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:11.387627    2580 scope.go:117] "RemoveContainer" containerID="518b454335eec3fe03d385f04954782ad6ed130f47baf01e8255655783f970ec"
Sep 19 01:13:11.388983 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:11.388965    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m" event={"ID":"39355481-6d16-4553-961a-f03ca171e712","Type":"ContainerStarted","Data":"3d1b365a4af2231417d9b0ef5255e38e738299f57134a96a4945752ad4fb5889"}
Sep 19 01:13:11.390878 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:11.390857    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nh6vq" event={"ID":"538e796f-8768-4bf9-ab1c-c3631bb14317","Type":"ContainerStarted","Data":"a60896277a47c9e18d5511ff17894a142ea3ccf379da992bdadb1bc2f73098ed"}
Sep 19 01:13:11.437667 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:11.437619    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-qxlvd" podStartSLOduration=8.095699247 podStartE2EDuration="16.43760527s" totalImagesPullingTime="8.341906023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:12:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:13:11.437025595 +0000 UTC m=+46.343887646" watchObservedRunningTime="2026-09-19 01:13:11.43760527 +0000 UTC m=+46.344467318"
Sep 19 01:13:11.489552 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:11.489481    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m" podStartSLOduration=9.511461787 podStartE2EDuration="17.489459077s" totalImagesPullingTime="7.97799729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:12:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:13:11.488872255 +0000 UTC m=+46.395734340" watchObservedRunningTime="2026-09-19 01:13:11.489459077 +0000 UTC m=+46.396321126"
Sep 19 01:13:11.561200 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:11.561150    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nh6vq" podStartSLOduration=36.208941485 podStartE2EDuration="44.56112818s" totalImagesPullingTime="8.352186695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:12:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:13:11.560429499 +0000 UTC m=+46.467291546" watchObservedRunningTime="2026-09-19 01:13:11.56112818 +0000 UTC m=+46.467990220"
Sep 19 01:13:11.562098 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:11.561850    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-44vdn" podStartSLOduration=8.572997011 podStartE2EDuration="16.561838922s" totalImagesPullingTime="7.988841911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:12:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:13:11.523573945 +0000 UTC m=+46.430435994" watchObservedRunningTime="2026-09-19 01:13:11.561838922 +0000 UTC m=+46.468700967"
Sep 19 01:13:11.777625 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:11.777559    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-bvc7f" podStartSLOduration=8.804978973 podStartE2EDuration="16.777535974s" totalImagesPullingTime="7.972557001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:12:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:13:11.655380721 +0000 UTC m=+46.562242769" watchObservedRunningTime="2026-09-19 01:13:11.777535974 +0000 UTC m=+46.684398023"
Sep 19 01:13:11.778665 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:11.778628    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-p4cpb" podStartSLOduration=3.788254552 podStartE2EDuration="11.77861413s" totalImagesPullingTime="7.990359578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:13:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:13:11.777513107 +0000 UTC m=+46.684375146" watchObservedRunningTime="2026-09-19 01:13:11.77861413 +0000 UTC m=+46.685476181"
Sep 19 01:13:11.977033 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:11.977004    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mdz52"]
Sep 19 01:13:11.980352 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:11.980330    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:11.983067 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:11.983047    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 19 01:13:11.983205 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:11.983129    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pqfkn\""
Sep 19 01:13:11.983281 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:11.983202    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 19 01:13:11.996705 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:11.996681    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mdz52"]
Sep 19 01:13:12.142474 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:12.142389    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk27r\" (UniqueName: \"kubernetes.io/projected/8830a56c-b4f4-4bbc-9303-52cd0b23d620-kube-api-access-jk27r\") pod \"insights-runtime-extractor-mdz52\" (UID: \"8830a56c-b4f4-4bbc-9303-52cd0b23d620\") " pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:12.142621 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:12.142570    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8830a56c-b4f4-4bbc-9303-52cd0b23d620-crio-socket\") pod \"insights-runtime-extractor-mdz52\" (UID: \"8830a56c-b4f4-4bbc-9303-52cd0b23d620\") " pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:12.142621 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:12.142602    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8830a56c-b4f4-4bbc-9303-52cd0b23d620-data-volume\") pod \"insights-runtime-extractor-mdz52\" (UID: \"8830a56c-b4f4-4bbc-9303-52cd0b23d620\") " pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:12.142695 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:12.142674    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8830a56c-b4f4-4bbc-9303-52cd0b23d620-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdz52\" (UID: \"8830a56c-b4f4-4bbc-9303-52cd0b23d620\") " pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:12.142729 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:12.142701    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8830a56c-b4f4-4bbc-9303-52cd0b23d620-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdz52\" (UID: \"8830a56c-b4f4-4bbc-9303-52cd0b23d620\") " pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:12.244051 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:12.244009    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8830a56c-b4f4-4bbc-9303-52cd0b23d620-crio-socket\") pod \"insights-runtime-extractor-mdz52\" (UID: \"8830a56c-b4f4-4bbc-9303-52cd0b23d620\") " pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:12.244219 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:12.244055    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8830a56c-b4f4-4bbc-9303-52cd0b23d620-data-volume\") pod \"insights-runtime-extractor-mdz52\" (UID: \"8830a56c-b4f4-4bbc-9303-52cd0b23d620\") " pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:12.244219 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:12.244104    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8830a56c-b4f4-4bbc-9303-52cd0b23d620-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdz52\" (UID: \"8830a56c-b4f4-4bbc-9303-52cd0b23d620\") " pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:12.244219 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:12.244129    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8830a56c-b4f4-4bbc-9303-52cd0b23d620-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdz52\" (UID: \"8830a56c-b4f4-4bbc-9303-52cd0b23d620\") " pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:12.244219 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:12.244149    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8830a56c-b4f4-4bbc-9303-52cd0b23d620-crio-socket\") pod \"insights-runtime-extractor-mdz52\" (UID: \"8830a56c-b4f4-4bbc-9303-52cd0b23d620\") " pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:12.244398 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:12.244231    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 01:13:12.244398 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:12.244244    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk27r\" (UniqueName: \"kubernetes.io/projected/8830a56c-b4f4-4bbc-9303-52cd0b23d620-kube-api-access-jk27r\") pod \"insights-runtime-extractor-mdz52\" (UID: \"8830a56c-b4f4-4bbc-9303-52cd0b23d620\") " pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:12.244398 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:12.244262    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 01:13:12.244398 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:12.244294    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8830a56c-b4f4-4bbc-9303-52cd0b23d620-kube-rbac-proxy-cm podName:8830a56c-b4f4-4bbc-9303-52cd0b23d620 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:12.744275273 +0000 UTC m=+47.651137313 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8830a56c-b4f4-4bbc-9303-52cd0b23d620-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mdz52" (UID: "8830a56c-b4f4-4bbc-9303-52cd0b23d620") : configmap "kube-rbac-proxy" not found
Sep 19 01:13:12.244398 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:12.244313    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8830a56c-b4f4-4bbc-9303-52cd0b23d620-insights-runtime-extractor-tls podName:8830a56c-b4f4-4bbc-9303-52cd0b23d620 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:12.744303307 +0000 UTC m=+47.651165340 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8830a56c-b4f4-4bbc-9303-52cd0b23d620-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mdz52" (UID: "8830a56c-b4f4-4bbc-9303-52cd0b23d620") : secret "insights-runtime-extractor-tls" not found
Sep 19 01:13:12.244663 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:12.244531    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8830a56c-b4f4-4bbc-9303-52cd0b23d620-data-volume\") pod \"insights-runtime-extractor-mdz52\" (UID: \"8830a56c-b4f4-4bbc-9303-52cd0b23d620\") " pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:12.259666 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:12.259642    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk27r\" (UniqueName: \"kubernetes.io/projected/8830a56c-b4f4-4bbc-9303-52cd0b23d620-kube-api-access-jk27r\") pod \"insights-runtime-extractor-mdz52\" (UID: \"8830a56c-b4f4-4bbc-9303-52cd0b23d620\") " pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:12.396878 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:12.396803    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f6bdh_74c8e2f6-1b01-4bd1-b491-ad407a15a109/console-operator/1.log"
Sep 19 01:13:12.397276 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:12.397246    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f6bdh_74c8e2f6-1b01-4bd1-b491-ad407a15a109/console-operator/0.log"
Sep 19 01:13:12.397342 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:12.397284    2580 generic.go:356] "Generic (PLEG): container finished" podID="74c8e2f6-1b01-4bd1-b491-ad407a15a109" containerID="0e6f423eefec792a0a78670e6b89b4f0343fb8e7b5aeec66257ce63ab4477713" exitCode=255
Sep 19 01:13:12.397410 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:12.397383    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh" event={"ID":"74c8e2f6-1b01-4bd1-b491-ad407a15a109","Type":"ContainerDied","Data":"0e6f423eefec792a0a78670e6b89b4f0343fb8e7b5aeec66257ce63ab4477713"}
Sep 19 01:13:12.397454 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:12.397440    2580 scope.go:117] "RemoveContainer" containerID="518b454335eec3fe03d385f04954782ad6ed130f47baf01e8255655783f970ec"
Sep 19 01:13:12.397638 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:12.397615    2580 scope.go:117] "RemoveContainer" containerID="0e6f423eefec792a0a78670e6b89b4f0343fb8e7b5aeec66257ce63ab4477713"
Sep 19 01:13:12.397763 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:12.397657    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:13:12.397826 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:12.397799    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-f6bdh_openshift-console-operator(74c8e2f6-1b01-4bd1-b491-ad407a15a109)\"" pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh" podUID="74c8e2f6-1b01-4bd1-b491-ad407a15a109"
Sep 19 01:13:12.750239 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:12.750207    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8830a56c-b4f4-4bbc-9303-52cd0b23d620-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdz52\" (UID: \"8830a56c-b4f4-4bbc-9303-52cd0b23d620\") " pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:12.750239 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:12.750241    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8830a56c-b4f4-4bbc-9303-52cd0b23d620-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdz52\" (UID: \"8830a56c-b4f4-4bbc-9303-52cd0b23d620\") " pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:12.750485 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:12.750342    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 01:13:12.750485 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:12.750418    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8830a56c-b4f4-4bbc-9303-52cd0b23d620-kube-rbac-proxy-cm podName:8830a56c-b4f4-4bbc-9303-52cd0b23d620 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:13.750399707 +0000 UTC m=+48.657261743 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8830a56c-b4f4-4bbc-9303-52cd0b23d620-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mdz52" (UID: "8830a56c-b4f4-4bbc-9303-52cd0b23d620") : configmap "kube-rbac-proxy" not found
Sep 19 01:13:12.750485 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:12.750348    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 01:13:12.750485 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:12.750467    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8830a56c-b4f4-4bbc-9303-52cd0b23d620-insights-runtime-extractor-tls podName:8830a56c-b4f4-4bbc-9303-52cd0b23d620 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:13.750457199 +0000 UTC m=+48.657319224 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8830a56c-b4f4-4bbc-9303-52cd0b23d620-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mdz52" (UID: "8830a56c-b4f4-4bbc-9303-52cd0b23d620") : secret "insights-runtime-extractor-tls" not found
Sep 19 01:13:13.402612 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:13.402581    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f6bdh_74c8e2f6-1b01-4bd1-b491-ad407a15a109/console-operator/1.log"
Sep 19 01:13:13.403050 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:13.402948    2580 scope.go:117] "RemoveContainer" containerID="0e6f423eefec792a0a78670e6b89b4f0343fb8e7b5aeec66257ce63ab4477713"
Sep 19 01:13:13.403125 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:13.403106    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-f6bdh_openshift-console-operator(74c8e2f6-1b01-4bd1-b491-ad407a15a109)\"" pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh" podUID="74c8e2f6-1b01-4bd1-b491-ad407a15a109"
Sep 19 01:13:13.761098 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:13.761067    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8830a56c-b4f4-4bbc-9303-52cd0b23d620-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdz52\" (UID: \"8830a56c-b4f4-4bbc-9303-52cd0b23d620\") " pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:13.761298 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:13.761103    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8830a56c-b4f4-4bbc-9303-52cd0b23d620-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdz52\" (UID: \"8830a56c-b4f4-4bbc-9303-52cd0b23d620\") " pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:13.761298 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:13.761202    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 01:13:13.761298 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:13.761267    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 01:13:13.761298 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:13.761277    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8830a56c-b4f4-4bbc-9303-52cd0b23d620-kube-rbac-proxy-cm podName:8830a56c-b4f4-4bbc-9303-52cd0b23d620 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:15.761260598 +0000 UTC m=+50.668122627 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8830a56c-b4f4-4bbc-9303-52cd0b23d620-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mdz52" (UID: "8830a56c-b4f4-4bbc-9303-52cd0b23d620") : configmap "kube-rbac-proxy" not found
Sep 19 01:13:13.761465 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:13.761317    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8830a56c-b4f4-4bbc-9303-52cd0b23d620-insights-runtime-extractor-tls podName:8830a56c-b4f4-4bbc-9303-52cd0b23d620 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:15.761302668 +0000 UTC m=+50.668164695 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8830a56c-b4f4-4bbc-9303-52cd0b23d620-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mdz52" (UID: "8830a56c-b4f4-4bbc-9303-52cd0b23d620") : secret "insights-runtime-extractor-tls" not found
Sep 19 01:13:14.674087 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:14.674061    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-grkk4_54557d74-f7ed-44ac-b483-9942019a9c4d/dns-node-resolver/0.log"
Sep 19 01:13:15.660403 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:15.660371    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dvd6m_2857e44c-032d-4586-b438-26ad712aef9c/node-ca/0.log"
Sep 19 01:13:15.776208 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:15.776176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8830a56c-b4f4-4bbc-9303-52cd0b23d620-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdz52\" (UID: \"8830a56c-b4f4-4bbc-9303-52cd0b23d620\") " pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:15.776208 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:15.776209    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8830a56c-b4f4-4bbc-9303-52cd0b23d620-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdz52\" (UID: \"8830a56c-b4f4-4bbc-9303-52cd0b23d620\") " pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:15.776713 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:15.776308    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 01:13:15.776713 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:15.776347    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 01:13:15.776713 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:15.776387    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8830a56c-b4f4-4bbc-9303-52cd0b23d620-kube-rbac-proxy-cm podName:8830a56c-b4f4-4bbc-9303-52cd0b23d620 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:19.77636644 +0000 UTC m=+54.683228481 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8830a56c-b4f4-4bbc-9303-52cd0b23d620-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mdz52" (UID: "8830a56c-b4f4-4bbc-9303-52cd0b23d620") : configmap "kube-rbac-proxy" not found
Sep 19 01:13:15.776713 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:15.776407    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8830a56c-b4f4-4bbc-9303-52cd0b23d620-insights-runtime-extractor-tls podName:8830a56c-b4f4-4bbc-9303-52cd0b23d620 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:19.776397957 +0000 UTC m=+54.683259984 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8830a56c-b4f4-4bbc-9303-52cd0b23d620-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mdz52" (UID: "8830a56c-b4f4-4bbc-9303-52cd0b23d620") : secret "insights-runtime-extractor-tls" not found
Sep 19 01:13:16.583014 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:16.582977    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-registry-tls\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:16.583178 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:16.583142    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 01:13:16.583178 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:16.583163    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw: secret "image-registry-tls" not found
Sep 19 01:13:16.583254 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:16.583189    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f921d091-7bf8-4d6d-81e4-6162afac0198-metrics-tls\") pod \"dns-default-v6ppc\" (UID: \"f921d091-7bf8-4d6d-81e4-6162afac0198\") " pod="openshift-dns/dns-default-v6ppc"
Sep 19 01:13:16.583254 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:16.583222    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-registry-tls podName:a439b5fa-a142-4815-b145-c3278eea25a0 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:32.58320418 +0000 UTC m=+67.490066223 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-registry-tls") pod "image-registry-5d7bd8c8fd-kxlmw" (UID: "a439b5fa-a142-4815-b145-c3278eea25a0") : secret "image-registry-tls" not found
Sep 19 01:13:16.583323 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:16.583286    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 01:13:16.583356 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:16.583330    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f921d091-7bf8-4d6d-81e4-6162afac0198-metrics-tls podName:f921d091-7bf8-4d6d-81e4-6162afac0198 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:32.583318269 +0000 UTC m=+67.490180299 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f921d091-7bf8-4d6d-81e4-6162afac0198-metrics-tls") pod "dns-default-v6ppc" (UID: "f921d091-7bf8-4d6d-81e4-6162afac0198") : secret "dns-default-metrics-tls" not found
Sep 19 01:13:16.659010 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:16.658982    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p4cpb_aa9e1f35-1684-4704-8bd7-532245d3fc97/serve-healthcheck-canary/0.log"
Sep 19 01:13:16.683697 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:16.683671    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/af99297d-b9c3-4e8d-b018-cb271f970733-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-f5gtc\" (UID: \"af99297d-b9c3-4e8d-b018-cb271f970733\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-f5gtc"
Sep 19 01:13:16.683845 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:16.683719    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c63ea3c5-742f-4077-a661-388d76e45d63-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-9bmsn\" (UID: \"c63ea3c5-742f-4077-a661-388d76e45d63\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9bmsn"
Sep 19 01:13:16.683845 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:16.683811    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 19 01:13:16.683930 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:16.683866    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 01:13:16.683930 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:16.683879    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af99297d-b9c3-4e8d-b018-cb271f970733-samples-operator-tls podName:af99297d-b9c3-4e8d-b018-cb271f970733 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:32.68386134 +0000 UTC m=+67.590723385 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/af99297d-b9c3-4e8d-b018-cb271f970733-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-f5gtc" (UID: "af99297d-b9c3-4e8d-b018-cb271f970733") : secret "samples-operator-tls" not found
Sep 19 01:13:16.683930 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:16.683904    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c63ea3c5-742f-4077-a661-388d76e45d63-cluster-monitoring-operator-tls podName:c63ea3c5-742f-4077-a661-388d76e45d63 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:32.683894076 +0000 UTC m=+67.590756113 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c63ea3c5-742f-4077-a661-388d76e45d63-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-9bmsn" (UID: "c63ea3c5-742f-4077-a661-388d76e45d63") : secret "cluster-monitoring-operator-tls" not found
Sep 19 01:13:16.784608 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:16.784581    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-metrics-certs\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:16.785002 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:16.784649    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cb55096-dad6-4d87-a63b-48e18dc7a011-service-ca-bundle\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:16.785002 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:16.784733    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 19 01:13:16.785002 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:16.784799    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-metrics-certs podName:9cb55096-dad6-4d87-a63b-48e18dc7a011 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:32.78478055 +0000 UTC m=+67.691642578 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-metrics-certs") pod "router-default-69d68c9d66-2gblx" (UID: "9cb55096-dad6-4d87-a63b-48e18dc7a011") : secret "router-metrics-certs-default" not found
Sep 19 01:13:16.785002 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:16.784813    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9cb55096-dad6-4d87-a63b-48e18dc7a011-service-ca-bundle podName:9cb55096-dad6-4d87-a63b-48e18dc7a011 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:32.78480735 +0000 UTC m=+67.691669376 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9cb55096-dad6-4d87-a63b-48e18dc7a011-service-ca-bundle") pod "router-default-69d68c9d66-2gblx" (UID: "9cb55096-dad6-4d87-a63b-48e18dc7a011") : configmap references non-existent config key: service-ca.crt
Sep 19 01:13:17.269428 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:17.269401    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-bvc7f_2c41153e-4f57-4158-83b8-18563e2750e8/kube-storage-version-migrator-operator/0.log"
Sep 19 01:13:19.809648 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:19.809617    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8830a56c-b4f4-4bbc-9303-52cd0b23d620-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdz52\" (UID: \"8830a56c-b4f4-4bbc-9303-52cd0b23d620\") " pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:19.809648 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:19.809652    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8830a56c-b4f4-4bbc-9303-52cd0b23d620-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdz52\" (UID: \"8830a56c-b4f4-4bbc-9303-52cd0b23d620\") " pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:19.810049 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:19.809750    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 01:13:19.810049 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:19.809812    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8830a56c-b4f4-4bbc-9303-52cd0b23d620-insights-runtime-extractor-tls podName:8830a56c-b4f4-4bbc-9303-52cd0b23d620 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:27.809799873 +0000 UTC m=+62.716661899 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8830a56c-b4f4-4bbc-9303-52cd0b23d620-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mdz52" (UID: "8830a56c-b4f4-4bbc-9303-52cd0b23d620") : secret "insights-runtime-extractor-tls" not found
Sep 19 01:13:19.810184 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:19.810165    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8830a56c-b4f4-4bbc-9303-52cd0b23d620-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdz52\" (UID: \"8830a56c-b4f4-4bbc-9303-52cd0b23d620\") " pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:21.105962 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:21.105933    2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh"
Sep 19 01:13:21.106299 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:21.105982    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh"
Sep 19 01:13:21.106348 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:21.106304    2580 scope.go:117] "RemoveContainer" containerID="0e6f423eefec792a0a78670e6b89b4f0343fb8e7b5aeec66257ce63ab4477713"
Sep 19 01:13:21.106459 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:21.106441    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-f6bdh_openshift-console-operator(74c8e2f6-1b01-4bd1-b491-ad407a15a109)\"" pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh" podUID="74c8e2f6-1b01-4bd1-b491-ad407a15a109"
Sep 19 01:13:26.297129 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:26.297093    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7z77w"
Sep 19 01:13:27.871178 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:27.871140    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8830a56c-b4f4-4bbc-9303-52cd0b23d620-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdz52\" (UID: \"8830a56c-b4f4-4bbc-9303-52cd0b23d620\") " pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:27.873440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:27.873408    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8830a56c-b4f4-4bbc-9303-52cd0b23d620-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdz52\" (UID: \"8830a56c-b4f4-4bbc-9303-52cd0b23d620\") " pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:27.892062 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:27.892044    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pqfkn\""
Sep 19 01:13:27.900138 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:27.900120    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mdz52"
Sep 19 01:13:28.171303 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:28.171116    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mdz52"]
Sep 19 01:13:28.173703 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:13:28.173680    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8830a56c_b4f4_4bbc_9303_52cd0b23d620.slice/crio-345b6b9576390138c2abf223ee73277bd3a0647abc9f0bd7cc2e05f67db8ac0b WatchSource:0}: Error finding container 345b6b9576390138c2abf223ee73277bd3a0647abc9f0bd7cc2e05f67db8ac0b: Status 404 returned error can't find the container with id 345b6b9576390138c2abf223ee73277bd3a0647abc9f0bd7cc2e05f67db8ac0b
Sep 19 01:13:28.450418 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:28.450339    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mdz52" event={"ID":"8830a56c-b4f4-4bbc-9303-52cd0b23d620","Type":"ContainerStarted","Data":"8d988bff0603888fa7820651d44095e3a8afa0bae0d3203f2841b68e243a31bf"}
Sep 19 01:13:28.450418 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:28.450394    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mdz52" event={"ID":"8830a56c-b4f4-4bbc-9303-52cd0b23d620","Type":"ContainerStarted","Data":"345b6b9576390138c2abf223ee73277bd3a0647abc9f0bd7cc2e05f67db8ac0b"}
Sep 19 01:13:29.455842 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:29.455806    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mdz52" event={"ID":"8830a56c-b4f4-4bbc-9303-52cd0b23d620","Type":"ContainerStarted","Data":"453f048cb1673dccb491579a967489938480484f4dda26634aa7e49948cfcbdb"}
Sep 19 01:13:30.460986 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:30.460955    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mdz52" event={"ID":"8830a56c-b4f4-4bbc-9303-52cd0b23d620","Type":"ContainerStarted","Data":"c8e7ee7375f614622f44614b9074530c96f3dce9cb89a55635c13e04a30442a0"}
Sep 19 01:13:30.480881 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:30.480842    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mdz52" podStartSLOduration=17.499785548 podStartE2EDuration="19.48082682s" totalImagesPullingTime="1.981041272s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 01:13:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:13:30.4800092 +0000 UTC m=+65.386871246" watchObservedRunningTime="2026-09-19 01:13:30.48082682 +0000 UTC m=+65.387688867"
Sep 19 01:13:32.607438 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.607391    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f921d091-7bf8-4d6d-81e4-6162afac0198-metrics-tls\") pod \"dns-default-v6ppc\" (UID: \"f921d091-7bf8-4d6d-81e4-6162afac0198\") " pod="openshift-dns/dns-default-v6ppc"
Sep 19 01:13:32.607905 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.607487    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-registry-tls\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:32.609893 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.609868    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f921d091-7bf8-4d6d-81e4-6162afac0198-metrics-tls\") pod \"dns-default-v6ppc\" (UID: \"f921d091-7bf8-4d6d-81e4-6162afac0198\") " pod="openshift-dns/dns-default-v6ppc"
Sep 19 01:13:32.609961 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.609940    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-registry-tls\") pod \"image-registry-5d7bd8c8fd-kxlmw\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") " pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:32.708575 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.708531    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/af99297d-b9c3-4e8d-b018-cb271f970733-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-f5gtc\" (UID: \"af99297d-b9c3-4e8d-b018-cb271f970733\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-f5gtc"
Sep 19 01:13:32.708575 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.708589    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c63ea3c5-742f-4077-a661-388d76e45d63-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-9bmsn\" (UID: \"c63ea3c5-742f-4077-a661-388d76e45d63\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9bmsn"
Sep 19 01:13:32.710929 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.710903    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/af99297d-b9c3-4e8d-b018-cb271f970733-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-f5gtc\" (UID: \"af99297d-b9c3-4e8d-b018-cb271f970733\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-f5gtc"
Sep 19 01:13:32.711047 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.710985    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c63ea3c5-742f-4077-a661-388d76e45d63-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-9bmsn\" (UID: \"c63ea3c5-742f-4077-a661-388d76e45d63\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9bmsn"
Sep 19 01:13:32.809050 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.808994    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-metrics-certs\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:32.809230 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.809070    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cb55096-dad6-4d87-a63b-48e18dc7a011-service-ca-bundle\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:32.809676 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.809657    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cb55096-dad6-4d87-a63b-48e18dc7a011-service-ca-bundle\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:32.811286 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.811266    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cb55096-dad6-4d87-a63b-48e18dc7a011-metrics-certs\") pod \"router-default-69d68c9d66-2gblx\" (UID: \"9cb55096-dad6-4d87-a63b-48e18dc7a011\") " pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:32.816846 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.816827    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5r66s\""
Sep 19 01:13:32.824928 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.824912    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:32.855008 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.854983    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h9l7c\""
Sep 19 01:13:32.863168 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.863097    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v6ppc"
Sep 19 01:13:32.910280 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.910240    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01b79542-9c56-4f26-a1d4-c4f2257ff152-metrics-certs\") pod \"network-metrics-daemon-hpjz8\" (UID: \"01b79542-9c56-4f26-a1d4-c4f2257ff152\") " pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:13:32.910280 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.910281    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2cd09385-9d78-428c-bb46-efeb37e8e55a-original-pull-secret\") pod \"global-pull-secret-syncer-kj96s\" (UID: \"2cd09385-9d78-428c-bb46-efeb37e8e55a\") " pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:13:32.912875 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.912855    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 19 01:13:32.913658 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.913599    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 19 01:13:32.923088 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.922832    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-4rbzb\""
Sep 19 01:13:32.924444 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.924376    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01b79542-9c56-4f26-a1d4-c4f2257ff152-metrics-certs\") pod \"network-metrics-daemon-hpjz8\" (UID: \"01b79542-9c56-4f26-a1d4-c4f2257ff152\") " pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:13:32.925589 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.925555    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2cd09385-9d78-428c-bb46-efeb37e8e55a-original-pull-secret\") pod \"global-pull-secret-syncer-kj96s\" (UID: \"2cd09385-9d78-428c-bb46-efeb37e8e55a\") " pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:13:32.931540 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.931522    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-f5gtc"
Sep 19 01:13:32.970664 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.970636    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lthdr\""
Sep 19 01:13:32.971746 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.971689    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj96s"
Sep 19 01:13:32.972001 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.971984    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-wbbj7\""
Sep 19 01:13:32.978428 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.977294    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hpjz8"
Sep 19 01:13:32.980254 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:32.980224    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9bmsn"
Sep 19 01:13:33.026124 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:33.025957    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-gsrf7\""
Sep 19 01:13:33.033781 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:33.033459    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:33.101888 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:33.101857    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"]
Sep 19 01:13:33.106442 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:13:33.106419    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda439b5fa_a142_4815_b145_c3278eea25a0.slice/crio-8ed805986ed4f74963e2917a0acf88bb76c1778c28a0d6ba4e5cde1bce215048 WatchSource:0}: Error finding container 8ed805986ed4f74963e2917a0acf88bb76c1778c28a0d6ba4e5cde1bce215048: Status 404 returned error can't find the container with id 8ed805986ed4f74963e2917a0acf88bb76c1778c28a0d6ba4e5cde1bce215048
Sep 19 01:13:33.140057 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:33.140033    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v6ppc"]
Sep 19 01:13:33.143139 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:13:33.143119    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf921d091_7bf8_4d6d_81e4_6162afac0198.slice/crio-0e68acffb2db8ed7bcde05e58680efc870bb0d8daec9b657e2f13ac31b4f2e17 WatchSource:0}: Error finding container 0e68acffb2db8ed7bcde05e58680efc870bb0d8daec9b657e2f13ac31b4f2e17: Status 404 returned error can't find the container with id 0e68acffb2db8ed7bcde05e58680efc870bb0d8daec9b657e2f13ac31b4f2e17
Sep 19 01:13:33.205882 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:33.205854    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-f5gtc"]
Sep 19 01:13:33.257171 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:33.257143    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9bmsn"]
Sep 19 01:13:33.261199 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:33.261175    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hpjz8"]
Sep 19 01:13:33.261652 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:13:33.261630    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc63ea3c5_742f_4077_a661_388d76e45d63.slice/crio-2c411e64a54b641e0c88d76f595d688ae1b0224615d5101f4a25b666c5f99fce WatchSource:0}: Error finding container 2c411e64a54b641e0c88d76f595d688ae1b0224615d5101f4a25b666c5f99fce: Status 404 returned error can't find the container with id 2c411e64a54b641e0c88d76f595d688ae1b0224615d5101f4a25b666c5f99fce
Sep 19 01:13:33.264600 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:13:33.264573    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01b79542_9c56_4f26_a1d4_c4f2257ff152.slice/crio-78c05e7a4611cc5d792c6fe5bd978b1ff140decb3c55a564cd0d6bf2e4f5ba75 WatchSource:0}: Error finding container 78c05e7a4611cc5d792c6fe5bd978b1ff140decb3c55a564cd0d6bf2e4f5ba75: Status 404 returned error can't find the container with id 78c05e7a4611cc5d792c6fe5bd978b1ff140decb3c55a564cd0d6bf2e4f5ba75
Sep 19 01:13:33.269279 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:33.269255    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kj96s"]
Sep 19 01:13:33.271704 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:13:33.271661    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cd09385_9d78_428c_bb46_efeb37e8e55a.slice/crio-f41e7227b714d093b7f73daed18d05f3284b7573e91f491e8a93af4c57d68dc0 WatchSource:0}: Error finding container f41e7227b714d093b7f73daed18d05f3284b7573e91f491e8a93af4c57d68dc0: Status 404 returned error can't find the container with id f41e7227b714d093b7f73daed18d05f3284b7573e91f491e8a93af4c57d68dc0
Sep 19 01:13:33.318661 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:33.318631    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-69d68c9d66-2gblx"]
Sep 19 01:13:33.321912 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:13:33.321889    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cb55096_dad6_4d87_a63b_48e18dc7a011.slice/crio-99c9b34f2e6be996e8391fae9186351a2282f9ec31e02cc5406be1b5cd5197ca WatchSource:0}: Error finding container 99c9b34f2e6be996e8391fae9186351a2282f9ec31e02cc5406be1b5cd5197ca: Status 404 returned error can't find the container with id 99c9b34f2e6be996e8391fae9186351a2282f9ec31e02cc5406be1b5cd5197ca
Sep 19 01:13:33.472150 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:33.472115    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw" event={"ID":"a439b5fa-a142-4815-b145-c3278eea25a0","Type":"ContainerStarted","Data":"5ed45ac5e3e58f6d83bbf8608c0b1e166132481fa608bb032128c8f1a430b9c0"}
Sep 19 01:13:33.472326 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:33.472168    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw" event={"ID":"a439b5fa-a142-4815-b145-c3278eea25a0","Type":"ContainerStarted","Data":"8ed805986ed4f74963e2917a0acf88bb76c1778c28a0d6ba4e5cde1bce215048"}
Sep 19 01:13:33.472326 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:33.472199    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:33.473771 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:33.473737    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-69d68c9d66-2gblx" event={"ID":"9cb55096-dad6-4d87-a63b-48e18dc7a011","Type":"ContainerStarted","Data":"c0f9f5e38ead20cb2b3abf1ad4e03816200e947decfa6875622913110edf2c82"}
Sep 19 01:13:33.473892 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:33.473788    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-69d68c9d66-2gblx" event={"ID":"9cb55096-dad6-4d87-a63b-48e18dc7a011","Type":"ContainerStarted","Data":"99c9b34f2e6be996e8391fae9186351a2282f9ec31e02cc5406be1b5cd5197ca"}
Sep 19 01:13:33.475005 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:33.474977    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9bmsn" event={"ID":"c63ea3c5-742f-4077-a661-388d76e45d63","Type":"ContainerStarted","Data":"2c411e64a54b641e0c88d76f595d688ae1b0224615d5101f4a25b666c5f99fce"}
Sep 19 01:13:33.476295 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:33.476274    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-f5gtc" event={"ID":"af99297d-b9c3-4e8d-b018-cb271f970733","Type":"ContainerStarted","Data":"dde383c1ee47f9af2909071d16007dd5140448b1b4142f00861daff84323845d"}
Sep 19 01:13:33.477443 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:33.477424    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hpjz8" event={"ID":"01b79542-9c56-4f26-a1d4-c4f2257ff152","Type":"ContainerStarted","Data":"78c05e7a4611cc5d792c6fe5bd978b1ff140decb3c55a564cd0d6bf2e4f5ba75"}
Sep 19 01:13:33.478635 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:33.478616    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kj96s" event={"ID":"2cd09385-9d78-428c-bb46-efeb37e8e55a","Type":"ContainerStarted","Data":"f41e7227b714d093b7f73daed18d05f3284b7573e91f491e8a93af4c57d68dc0"}
Sep 19 01:13:33.479723 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:33.479701    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v6ppc" event={"ID":"f921d091-7bf8-4d6d-81e4-6162afac0198","Type":"ContainerStarted","Data":"0e68acffb2db8ed7bcde05e58680efc870bb0d8daec9b657e2f13ac31b4f2e17"}
Sep 19 01:13:33.494334 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:33.494204    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw" podStartSLOduration=45.494189812 podStartE2EDuration="45.494189812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:12:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:13:33.493707108 +0000 UTC m=+68.400569155" watchObservedRunningTime="2026-09-19 01:13:33.494189812 +0000 UTC m=+68.401051860"
Sep 19 01:13:33.518746 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:33.518694    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-69d68c9d66-2gblx" podStartSLOduration=38.518677733 podStartE2EDuration="38.518677733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:12:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:13:33.517860672 +0000 UTC m=+68.424722721" watchObservedRunningTime="2026-09-19 01:13:33.518677733 +0000 UTC m=+68.425539782"
Sep 19 01:13:34.035192 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:34.035112    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:34.038533 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:34.038469    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:34.057329 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:34.057171    2580 scope.go:117] "RemoveContainer" containerID="0e6f423eefec792a0a78670e6b89b4f0343fb8e7b5aeec66257ce63ab4477713"
Sep 19 01:13:34.490701 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:34.490677    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f6bdh_74c8e2f6-1b01-4bd1-b491-ad407a15a109/console-operator/1.log"
Sep 19 01:13:34.491572 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:34.491543    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh" event={"ID":"74c8e2f6-1b01-4bd1-b491-ad407a15a109","Type":"ContainerStarted","Data":"0f2a015fc19cd1c077cfde3b60e37467325d80721bfdd78e7069693c61f7cb13"}
Sep 19 01:13:34.491702 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:34.491616    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:34.492156 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:34.492135    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh"
Sep 19 01:13:34.493791 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:34.493694    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-69d68c9d66-2gblx"
Sep 19 01:13:34.514451 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:34.514104    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh" podStartSLOduration=31.532149553 podStartE2EDuration="39.514085642s" totalImagesPullingTime="7.981936089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:12:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:13:34.512740169 +0000 UTC m=+69.419602221" watchObservedRunningTime="2026-09-19 01:13:34.514085642 +0000 UTC m=+69.420947695"
Sep 19 01:13:34.633433 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:34.633371    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-fd4bb46c-f6bdh"
Sep 19 01:13:35.497580 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:35.497537    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hpjz8" event={"ID":"01b79542-9c56-4f26-a1d4-c4f2257ff152","Type":"ContainerStarted","Data":"e765654b9515314b7eb473b15d98daa6c987216e0795d0bc7ff06d3dbf285a2b"}
Sep 19 01:13:37.508829 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:37.508785    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9bmsn" event={"ID":"c63ea3c5-742f-4077-a661-388d76e45d63","Type":"ContainerStarted","Data":"d876be10d46d3223398c1fdb5e0bc74000d2b1eff5365d0240c1873fe4eda55e"}
Sep 19 01:13:37.510738 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:37.510710    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-f5gtc" event={"ID":"af99297d-b9c3-4e8d-b018-cb271f970733","Type":"ContainerStarted","Data":"e5fff11ee0bdb6328e51ef601c59547f95e01e61fe3668ee6739fdfab32ef938"}
Sep 19 01:13:37.513131 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:37.513104    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hpjz8" event={"ID":"01b79542-9c56-4f26-a1d4-c4f2257ff152","Type":"ContainerStarted","Data":"a158acdd10889e01d70f655b93bd2b165775ebf5ae665b06f7eaeb185ca74d19"}
Sep 19 01:13:37.514895 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:37.514872    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v6ppc" event={"ID":"f921d091-7bf8-4d6d-81e4-6162afac0198","Type":"ContainerStarted","Data":"df0d0f4b9e059eeae3b6e4d05fed233111f1b1e0106ad526ab1bef3b2124bfe3"}
Sep 19 01:13:37.530818 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:37.528952    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9bmsn" podStartSLOduration=39.193763839 podStartE2EDuration="42.528934495s" totalImagesPullingTime="3.335170656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:12:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:13:37.526088997 +0000 UTC m=+72.432951047" watchObservedRunningTime="2026-09-19 01:13:37.528934495 +0000 UTC m=+72.435796545"
Sep 19 01:13:37.543291 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:37.543245    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hpjz8" podStartSLOduration=68.985926077 podStartE2EDuration="1m10.54322659s" totalImagesPullingTime="1.557300513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:12:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:13:37.543169332 +0000 UTC m=+72.450031381" watchObservedRunningTime="2026-09-19 01:13:37.54322659 +0000 UTC m=+72.450088640"
Sep 19 01:13:38.520709 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:38.520684    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-f5gtc" event={"ID":"af99297d-b9c3-4e8d-b018-cb271f970733","Type":"ContainerStarted","Data":"5d6ca7e2e3316633532f4b1265060a61a4be62201c65e7d4e97b70eaeb8c8dbe"}
Sep 19 01:13:38.522390 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:38.522365    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v6ppc" event={"ID":"f921d091-7bf8-4d6d-81e4-6162afac0198","Type":"ContainerStarted","Data":"e7d85f8b8d391fc0f464dbcd40461d798d66394defe459a76aec893ea1215d36"}
Sep 19 01:13:38.561121 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:38.559433    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-v6ppc" podStartSLOduration=35.102781367 podStartE2EDuration="38.559412129s" totalImagesPullingTime="3.456630762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:13:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:13:38.55893052 +0000 UTC m=+73.465792569" watchObservedRunningTime="2026-09-19 01:13:38.559412129 +0000 UTC m=+73.466274181"
Sep 19 01:13:38.581752 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:38.581695    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-f5gtc" podStartSLOduration=40.228410202 podStartE2EDuration="43.581670842s" totalImagesPullingTime="3.35326064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:12:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:13:38.581181961 +0000 UTC m=+73.488044010" watchObservedRunningTime="2026-09-19 01:13:38.581670842 +0000 UTC m=+73.488533089"
Sep 19 01:13:39.528449 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:39.528412    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kj96s" event={"ID":"2cd09385-9d78-428c-bb46-efeb37e8e55a","Type":"ContainerStarted","Data":"b4295f23785b703ed4d5efef71d930d950c8e0132226260a8e5537788d29ed0c"}
Sep 19 01:13:39.528933 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:39.528473    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-v6ppc"
Sep 19 01:13:39.559241 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:39.559191    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-kj96s" podStartSLOduration=67.319278183 podStartE2EDuration="1m12.559174761s" totalImagesPullingTime="5.239896578s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 01:12:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:13:39.558876419 +0000 UTC m=+74.465738481" watchObservedRunningTime="2026-09-19 01:13:39.559174761 +0000 UTC m=+74.466036809"
Sep 19 01:13:43.405483 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:43.405456    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nh6vq"
Sep 19 01:13:46.534840 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:46.534813    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-v6ppc"
Sep 19 01:13:50.356846 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:50.356813    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-5ld7x"]
Sep 19 01:13:50.361431 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:50.361413    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-5ld7x"
Sep 19 01:13:50.363978 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:50.363954    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 19 01:13:50.364106 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:50.364016    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-2fsbt\""
Sep 19 01:13:50.364262 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:50.364248    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 19 01:13:50.364992 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:50.364979    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 19 01:13:50.372473 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:50.372450    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-5ld7x"]
Sep 19 01:13:50.443691 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:50.443661    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8lsp\" (UniqueName: \"kubernetes.io/projected/b3c58358-5b1f-4617-9945-42246f59e5f7-kube-api-access-d8lsp\") pod \"prometheus-operator-7bd5d56d6-5ld7x\" (UID: \"b3c58358-5b1f-4617-9945-42246f59e5f7\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-5ld7x"
Sep 19 01:13:50.443816 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:50.443694    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3c58358-5b1f-4617-9945-42246f59e5f7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-5ld7x\" (UID: \"b3c58358-5b1f-4617-9945-42246f59e5f7\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-5ld7x"
Sep 19 01:13:50.443816 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:50.443718    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3c58358-5b1f-4617-9945-42246f59e5f7-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-5ld7x\" (UID: \"b3c58358-5b1f-4617-9945-42246f59e5f7\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-5ld7x"
Sep 19 01:13:50.443816 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:50.443744    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3c58358-5b1f-4617-9945-42246f59e5f7-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-5ld7x\" (UID: \"b3c58358-5b1f-4617-9945-42246f59e5f7\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-5ld7x"
Sep 19 01:13:50.544481 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:50.544452    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3c58358-5b1f-4617-9945-42246f59e5f7-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-5ld7x\" (UID: \"b3c58358-5b1f-4617-9945-42246f59e5f7\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-5ld7x"
Sep 19 01:13:50.544613 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:50.544515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8lsp\" (UniqueName: \"kubernetes.io/projected/b3c58358-5b1f-4617-9945-42246f59e5f7-kube-api-access-d8lsp\") pod \"prometheus-operator-7bd5d56d6-5ld7x\" (UID: \"b3c58358-5b1f-4617-9945-42246f59e5f7\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-5ld7x"
Sep 19 01:13:50.544613 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:50.544536    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3c58358-5b1f-4617-9945-42246f59e5f7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-5ld7x\" (UID: \"b3c58358-5b1f-4617-9945-42246f59e5f7\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-5ld7x"
Sep 19 01:13:50.544613 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:50.544559    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3c58358-5b1f-4617-9945-42246f59e5f7-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-5ld7x\" (UID: \"b3c58358-5b1f-4617-9945-42246f59e5f7\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-5ld7x"
Sep 19 01:13:50.544613 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:50.544600    2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 19 01:13:50.544806 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:50.544669    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3c58358-5b1f-4617-9945-42246f59e5f7-prometheus-operator-tls podName:b3c58358-5b1f-4617-9945-42246f59e5f7 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:51.044653109 +0000 UTC m=+85.951515139 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/b3c58358-5b1f-4617-9945-42246f59e5f7-prometheus-operator-tls") pod "prometheus-operator-7bd5d56d6-5ld7x" (UID: "b3c58358-5b1f-4617-9945-42246f59e5f7") : secret "prometheus-operator-tls" not found
Sep 19 01:13:50.545185 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:50.545167    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3c58358-5b1f-4617-9945-42246f59e5f7-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-5ld7x\" (UID: \"b3c58358-5b1f-4617-9945-42246f59e5f7\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-5ld7x"
Sep 19 01:13:50.547023 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:50.547001    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3c58358-5b1f-4617-9945-42246f59e5f7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-5ld7x\" (UID: \"b3c58358-5b1f-4617-9945-42246f59e5f7\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-5ld7x"
Sep 19 01:13:50.553466 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:50.553445    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8lsp\" (UniqueName: \"kubernetes.io/projected/b3c58358-5b1f-4617-9945-42246f59e5f7-kube-api-access-d8lsp\") pod \"prometheus-operator-7bd5d56d6-5ld7x\" (UID: \"b3c58358-5b1f-4617-9945-42246f59e5f7\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-5ld7x"
Sep 19 01:13:51.049388 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:51.049345    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3c58358-5b1f-4617-9945-42246f59e5f7-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-5ld7x\" (UID: \"b3c58358-5b1f-4617-9945-42246f59e5f7\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-5ld7x"
Sep 19 01:13:51.049557 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:51.049489    2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 19 01:13:51.049617 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:51.049585    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3c58358-5b1f-4617-9945-42246f59e5f7-prometheus-operator-tls podName:b3c58358-5b1f-4617-9945-42246f59e5f7 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:52.04956899 +0000 UTC m=+86.956431017 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/b3c58358-5b1f-4617-9945-42246f59e5f7-prometheus-operator-tls") pod "prometheus-operator-7bd5d56d6-5ld7x" (UID: "b3c58358-5b1f-4617-9945-42246f59e5f7") : secret "prometheus-operator-tls" not found
Sep 19 01:13:52.059017 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:52.058983    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3c58358-5b1f-4617-9945-42246f59e5f7-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-5ld7x\" (UID: \"b3c58358-5b1f-4617-9945-42246f59e5f7\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-5ld7x"
Sep 19 01:13:52.061458 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:52.061435    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3c58358-5b1f-4617-9945-42246f59e5f7-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-5ld7x\" (UID: \"b3c58358-5b1f-4617-9945-42246f59e5f7\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-5ld7x"
Sep 19 01:13:52.170437 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:52.170397    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-5ld7x"
Sep 19 01:13:52.445792 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:52.445770    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-5ld7x"]
Sep 19 01:13:52.448008 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:13:52.447975    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3c58358_5b1f_4617_9945_42246f59e5f7.slice/crio-c354f1fbae31c14f871fa3d4cc61aa1c7733b87d5a350a52dc17f170751dd495 WatchSource:0}: Error finding container c354f1fbae31c14f871fa3d4cc61aa1c7733b87d5a350a52dc17f170751dd495: Status 404 returned error can't find the container with id c354f1fbae31c14f871fa3d4cc61aa1c7733b87d5a350a52dc17f170751dd495
Sep 19 01:13:52.576309 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:52.576278    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-5ld7x" event={"ID":"b3c58358-5b1f-4617-9945-42246f59e5f7","Type":"ContainerStarted","Data":"c354f1fbae31c14f871fa3d4cc61aa1c7733b87d5a350a52dc17f170751dd495"}
Sep 19 01:13:54.496415 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:54.496389    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:13:54.586126 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:54.586093    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-5ld7x" event={"ID":"b3c58358-5b1f-4617-9945-42246f59e5f7","Type":"ContainerStarted","Data":"3ca9b0db3f099552c70a4617e877fb13a87a8e237c8130c315453d8e9ff27c34"}
Sep 19 01:13:54.586303 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:54.586153    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-5ld7x" event={"ID":"b3c58358-5b1f-4617-9945-42246f59e5f7","Type":"ContainerStarted","Data":"82af662a7714913929a1d58f93e9cec06d40f358efd8d2c80eed3ac4f0b910e2"}
Sep 19 01:13:54.618328 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:54.618280    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-5ld7x" podStartSLOduration=3.36614361 podStartE2EDuration="4.618262791s" totalImagesPullingTime="1.252119181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:13:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:13:54.616995551 +0000 UTC m=+89.523857598" watchObservedRunningTime="2026-09-19 01:13:54.618262791 +0000 UTC m=+89.525124839"
Sep 19 01:13:56.709371 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.709338    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-86gqf"]
Sep 19 01:13:56.712727 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.712711    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-86gqf"
Sep 19 01:13:56.716239 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.716213    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 19 01:13:56.716409 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.716341    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 19 01:13:56.716483 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.716437    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-2nqjp\""
Sep 19 01:13:56.722454 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.722435    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-5q7kn"]
Sep 19 01:13:56.725385 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.725370    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn"
Sep 19 01:13:56.730003 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.729978    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 19 01:13:56.730121 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.730105    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 19 01:13:56.730203 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.730186    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 19 01:13:56.730262 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.730191    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-jbppp\""
Sep 19 01:13:56.733843 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.733821    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-86gqf"]
Sep 19 01:13:56.741317 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.741298    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-5q7kn"]
Sep 19 01:13:56.792216 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.792189    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qw5d\" (UniqueName: \"kubernetes.io/projected/551bcebb-cd3c-4ca5-ac56-54c6551248b0-kube-api-access-6qw5d\") pod \"kube-state-metrics-84c885658c-5q7kn\" (UID: \"551bcebb-cd3c-4ca5-ac56-54c6551248b0\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn"
Sep 19 01:13:56.792341 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.792221    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/31e2d5af-7ffd-4089-8628-eca0941659a9-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-86gqf\" (UID: \"31e2d5af-7ffd-4089-8628-eca0941659a9\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-86gqf"
Sep 19 01:13:56.792341 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.792240    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/31e2d5af-7ffd-4089-8628-eca0941659a9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-86gqf\" (UID: \"31e2d5af-7ffd-4089-8628-eca0941659a9\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-86gqf"
Sep 19 01:13:56.792341 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.792263    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/551bcebb-cd3c-4ca5-ac56-54c6551248b0-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-5q7kn\" (UID: \"551bcebb-cd3c-4ca5-ac56-54c6551248b0\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn"
Sep 19 01:13:56.792341 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.792328    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/551bcebb-cd3c-4ca5-ac56-54c6551248b0-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-5q7kn\" (UID: \"551bcebb-cd3c-4ca5-ac56-54c6551248b0\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn"
Sep 19 01:13:56.792482 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.792368    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/551bcebb-cd3c-4ca5-ac56-54c6551248b0-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-5q7kn\" (UID: \"551bcebb-cd3c-4ca5-ac56-54c6551248b0\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn"
Sep 19 01:13:56.792572 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.792385    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/31e2d5af-7ffd-4089-8628-eca0941659a9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-86gqf\" (UID: \"31e2d5af-7ffd-4089-8628-eca0941659a9\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-86gqf"
Sep 19 01:13:56.796446 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.792922    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/551bcebb-cd3c-4ca5-ac56-54c6551248b0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-5q7kn\" (UID: \"551bcebb-cd3c-4ca5-ac56-54c6551248b0\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn"
Sep 19 01:13:56.796446 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.793002    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/551bcebb-cd3c-4ca5-ac56-54c6551248b0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-5q7kn\" (UID: \"551bcebb-cd3c-4ca5-ac56-54c6551248b0\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn"
Sep 19 01:13:56.796446 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.793042    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc2xn\" (UniqueName: \"kubernetes.io/projected/31e2d5af-7ffd-4089-8628-eca0941659a9-kube-api-access-cc2xn\") pod \"openshift-state-metrics-86c5bc79d6-86gqf\" (UID: \"31e2d5af-7ffd-4089-8628-eca0941659a9\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-86gqf"
Sep 19 01:13:56.811251 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.811228    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8kpmp"]
Sep 19 01:13:56.814569 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.814554    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.819606 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.819588    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 19 01:13:56.819930 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.819897    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 19 01:13:56.820118 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.820095    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hn4x7\""
Sep 19 01:13:56.820533 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.820409    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 19 01:13:56.894405 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.894378    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/31e2d5af-7ffd-4089-8628-eca0941659a9-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-86gqf\" (UID: \"31e2d5af-7ffd-4089-8628-eca0941659a9\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-86gqf"
Sep 19 01:13:56.894625 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.894410    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/31e2d5af-7ffd-4089-8628-eca0941659a9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-86gqf\" (UID: \"31e2d5af-7ffd-4089-8628-eca0941659a9\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-86gqf"
Sep 19 01:13:56.894625 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.894433    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/551bcebb-cd3c-4ca5-ac56-54c6551248b0-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-5q7kn\" (UID: \"551bcebb-cd3c-4ca5-ac56-54c6551248b0\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn"
Sep 19 01:13:56.894625 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.894451    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/551bcebb-cd3c-4ca5-ac56-54c6551248b0-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-5q7kn\" (UID: \"551bcebb-cd3c-4ca5-ac56-54c6551248b0\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn"
Sep 19 01:13:56.894625 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:56.894575    2580 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 19 01:13:56.894823 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:56.894635    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/551bcebb-cd3c-4ca5-ac56-54c6551248b0-kube-state-metrics-tls podName:551bcebb-cd3c-4ca5-ac56-54c6551248b0 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:57.394615773 +0000 UTC m=+92.301477814 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/551bcebb-cd3c-4ca5-ac56-54c6551248b0-kube-state-metrics-tls") pod "kube-state-metrics-84c885658c-5q7kn" (UID: "551bcebb-cd3c-4ca5-ac56-54c6551248b0") : secret "kube-state-metrics-tls" not found
Sep 19 01:13:56.894823 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.894656    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.894823 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.894707    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-node-exporter-accelerators-collector-config\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.894823 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.894738    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/551bcebb-cd3c-4ca5-ac56-54c6551248b0-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-5q7kn\" (UID: \"551bcebb-cd3c-4ca5-ac56-54c6551248b0\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn"
Sep 19 01:13:56.894823 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.894763    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/31e2d5af-7ffd-4089-8628-eca0941659a9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-86gqf\" (UID: \"31e2d5af-7ffd-4089-8628-eca0941659a9\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-86gqf"
Sep 19 01:13:56.894823 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.894790    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/551bcebb-cd3c-4ca5-ac56-54c6551248b0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-5q7kn\" (UID: \"551bcebb-cd3c-4ca5-ac56-54c6551248b0\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn"
Sep 19 01:13:56.894823 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.894820    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-root\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.895176 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.894855    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/551bcebb-cd3c-4ca5-ac56-54c6551248b0-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-5q7kn\" (UID: \"551bcebb-cd3c-4ca5-ac56-54c6551248b0\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn"
Sep 19 01:13:56.895176 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:56.894900    2580 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 19 01:13:56.895176 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.894973    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-metrics-client-ca\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.895176 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:56.894981    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31e2d5af-7ffd-4089-8628-eca0941659a9-openshift-state-metrics-tls podName:31e2d5af-7ffd-4089-8628-eca0941659a9 nodeName:}" failed. No retries permitted until 2026-09-19 01:13:57.394962335 +0000 UTC m=+92.301824378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/31e2d5af-7ffd-4089-8628-eca0941659a9-openshift-state-metrics-tls") pod "openshift-state-metrics-86c5bc79d6-86gqf" (UID: "31e2d5af-7ffd-4089-8628-eca0941659a9") : secret "openshift-state-metrics-tls" not found
Sep 19 01:13:56.895176 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.895035    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vqv9\" (UniqueName: \"kubernetes.io/projected/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-kube-api-access-6vqv9\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.895176 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.895066    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-sys\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.895176 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.895120    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-node-exporter-tls\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.895176 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.895157    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/551bcebb-cd3c-4ca5-ac56-54c6551248b0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-5q7kn\" (UID: \"551bcebb-cd3c-4ca5-ac56-54c6551248b0\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn"
Sep 19 01:13:56.895602 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.895180    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/31e2d5af-7ffd-4089-8628-eca0941659a9-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-86gqf\" (UID: \"31e2d5af-7ffd-4089-8628-eca0941659a9\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-86gqf"
Sep 19 01:13:56.895602 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.895191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc2xn\" (UniqueName: \"kubernetes.io/projected/31e2d5af-7ffd-4089-8628-eca0941659a9-kube-api-access-cc2xn\") pod \"openshift-state-metrics-86c5bc79d6-86gqf\" (UID: \"31e2d5af-7ffd-4089-8628-eca0941659a9\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-86gqf"
Sep 19 01:13:56.895602 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.895249    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-node-exporter-wtmp\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.895602 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.895287    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-node-exporter-textfile\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.895602 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.895359    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qw5d\" (UniqueName: \"kubernetes.io/projected/551bcebb-cd3c-4ca5-ac56-54c6551248b0-kube-api-access-6qw5d\") pod \"kube-state-metrics-84c885658c-5q7kn\" (UID: \"551bcebb-cd3c-4ca5-ac56-54c6551248b0\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn"
Sep 19 01:13:56.895602 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.895431    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/551bcebb-cd3c-4ca5-ac56-54c6551248b0-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-5q7kn\" (UID: \"551bcebb-cd3c-4ca5-ac56-54c6551248b0\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn"
Sep 19 01:13:56.895804 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.895601    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/551bcebb-cd3c-4ca5-ac56-54c6551248b0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-5q7kn\" (UID: \"551bcebb-cd3c-4ca5-ac56-54c6551248b0\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn"
Sep 19 01:13:56.897111 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.897090    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/31e2d5af-7ffd-4089-8628-eca0941659a9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-86gqf\" (UID: \"31e2d5af-7ffd-4089-8628-eca0941659a9\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-86gqf"
Sep 19 01:13:56.897573 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.897553    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/551bcebb-cd3c-4ca5-ac56-54c6551248b0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-5q7kn\" (UID: \"551bcebb-cd3c-4ca5-ac56-54c6551248b0\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn"
Sep 19 01:13:56.914916 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.914890    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qw5d\" (UniqueName: \"kubernetes.io/projected/551bcebb-cd3c-4ca5-ac56-54c6551248b0-kube-api-access-6qw5d\") pod \"kube-state-metrics-84c885658c-5q7kn\" (UID: \"551bcebb-cd3c-4ca5-ac56-54c6551248b0\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn"
Sep 19 01:13:56.915802 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.915780    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc2xn\" (UniqueName: \"kubernetes.io/projected/31e2d5af-7ffd-4089-8628-eca0941659a9-kube-api-access-cc2xn\") pod \"openshift-state-metrics-86c5bc79d6-86gqf\" (UID: \"31e2d5af-7ffd-4089-8628-eca0941659a9\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-86gqf"
Sep 19 01:13:56.996429 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.996400    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.996607 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.996448    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-node-exporter-accelerators-collector-config\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.996607 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.996574    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-root\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.996722 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.996605    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-metrics-client-ca\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.996722 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.996632    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vqv9\" (UniqueName: \"kubernetes.io/projected/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-kube-api-access-6vqv9\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.996722 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.996656    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-sys\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.996722 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.996682    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-root\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.996722 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.996687    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-node-exporter-tls\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.996955 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.996755    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-node-exporter-wtmp\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.996955 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.996791    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-node-exporter-textfile\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.996955 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.996799    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-sys\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.996955 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:56.996760    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 19 01:13:56.997147 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.996961    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-node-exporter-wtmp\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.997147 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:13:56.997007    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-node-exporter-tls podName:e43af3ba-8b7a-488a-98e1-0acd8e823a1f nodeName:}" failed. No retries permitted until 2026-09-19 01:13:57.49698833 +0000 UTC m=+92.403850379 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-node-exporter-tls") pod "node-exporter-8kpmp" (UID: "e43af3ba-8b7a-488a-98e1-0acd8e823a1f") : secret "node-exporter-tls" not found
Sep 19 01:13:56.997147 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.997088    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-node-exporter-textfile\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.997147 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.997096    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-node-exporter-accelerators-collector-config\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.997147 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.997135    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-metrics-client-ca\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:56.998950 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:56.998927    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:57.023334 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:57.023306    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vqv9\" (UniqueName: \"kubernetes.io/projected/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-kube-api-access-6vqv9\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:57.400478 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:57.400403    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/551bcebb-cd3c-4ca5-ac56-54c6551248b0-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-5q7kn\" (UID: \"551bcebb-cd3c-4ca5-ac56-54c6551248b0\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn"
Sep 19 01:13:57.400478 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:57.400443    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/31e2d5af-7ffd-4089-8628-eca0941659a9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-86gqf\" (UID: \"31e2d5af-7ffd-4089-8628-eca0941659a9\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-86gqf"
Sep 19 01:13:57.402652 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:57.402631    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/551bcebb-cd3c-4ca5-ac56-54c6551248b0-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-5q7kn\" (UID: \"551bcebb-cd3c-4ca5-ac56-54c6551248b0\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn"
Sep 19 01:13:57.402773 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:57.402664    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/31e2d5af-7ffd-4089-8628-eca0941659a9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-86gqf\" (UID: \"31e2d5af-7ffd-4089-8628-eca0941659a9\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-86gqf"
Sep 19 01:13:57.500989 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:57.500956    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-node-exporter-tls\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:57.503276 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:57.503191    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e43af3ba-8b7a-488a-98e1-0acd8e823a1f-node-exporter-tls\") pod \"node-exporter-8kpmp\" (UID: \"e43af3ba-8b7a-488a-98e1-0acd8e823a1f\") " pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:57.622188 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:57.622162    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-86gqf"
Sep 19 01:13:57.634221 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:57.634200    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn"
Sep 19 01:13:57.723065 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:57.723039    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8kpmp"
Sep 19 01:13:57.735026 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:13:57.735003    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode43af3ba_8b7a_488a_98e1_0acd8e823a1f.slice/crio-73aab0851946e7de57d6bae694b5b342fdea6418d4adfd4c7b2a1d1fbd71027e WatchSource:0}: Error finding container 73aab0851946e7de57d6bae694b5b342fdea6418d4adfd4c7b2a1d1fbd71027e: Status 404 returned error can't find the container with id 73aab0851946e7de57d6bae694b5b342fdea6418d4adfd4c7b2a1d1fbd71027e
Sep 19 01:13:57.915314 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:57.915284    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-86gqf"]
Sep 19 01:13:57.917452 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:13:57.917413    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31e2d5af_7ffd_4089_8628_eca0941659a9.slice/crio-400f79ad796390813c5b7a0f4d1bdbdb222699e989f858a8a65964ac763a479e WatchSource:0}: Error finding container 400f79ad796390813c5b7a0f4d1bdbdb222699e989f858a8a65964ac763a479e: Status 404 returned error can't find the container with id 400f79ad796390813c5b7a0f4d1bdbdb222699e989f858a8a65964ac763a479e
Sep 19 01:13:57.935127 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:57.935025    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-5q7kn"]
Sep 19 01:13:57.938555 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:13:57.938524    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod551bcebb_cd3c_4ca5_ac56_54c6551248b0.slice/crio-917a108e2fa501ec45e6f2a622fef5a2c2893bbef3082671b9b06082b3d5d683 WatchSource:0}: Error finding container 917a108e2fa501ec45e6f2a622fef5a2c2893bbef3082671b9b06082b3d5d683: Status 404 returned error can't find the container with id 917a108e2fa501ec45e6f2a622fef5a2c2893bbef3082671b9b06082b3d5d683
Sep 19 01:13:58.602946 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:58.602906    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn" event={"ID":"551bcebb-cd3c-4ca5-ac56-54c6551248b0","Type":"ContainerStarted","Data":"917a108e2fa501ec45e6f2a622fef5a2c2893bbef3082671b9b06082b3d5d683"}
Sep 19 01:13:58.605023 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:58.604991    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8kpmp" event={"ID":"e43af3ba-8b7a-488a-98e1-0acd8e823a1f","Type":"ContainerStarted","Data":"73aab0851946e7de57d6bae694b5b342fdea6418d4adfd4c7b2a1d1fbd71027e"}
Sep 19 01:13:58.608128 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:58.608096    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-86gqf" event={"ID":"31e2d5af-7ffd-4089-8628-eca0941659a9","Type":"ContainerStarted","Data":"7a533c31b238f43ec35b58ebc1b98de86e37d0d558550cda87118f1613c32b41"}
Sep 19 01:13:58.608244 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:58.608170    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-86gqf" event={"ID":"31e2d5af-7ffd-4089-8628-eca0941659a9","Type":"ContainerStarted","Data":"0ef13f15198c93c0c64d3f6f3b37afefb21b3b0e965bf4c097a80ca75d3891c6"}
Sep 19 01:13:58.608244 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:58.608198    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-86gqf" event={"ID":"31e2d5af-7ffd-4089-8628-eca0941659a9","Type":"ContainerStarted","Data":"400f79ad796390813c5b7a0f4d1bdbdb222699e989f858a8a65964ac763a479e"}
Sep 19 01:13:58.809818 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:58.809781    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"]
Sep 19 01:13:58.813980 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:58.813947    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:58.817821 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:58.817122    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 19 01:13:58.817821 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:58.817549    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 19 01:13:58.817821 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:58.817664    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-f5djoshj0e8mg\""
Sep 19 01:13:58.818075 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:58.817953    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-6kzr4\""
Sep 19 01:13:58.821671 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:58.820048    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 19 01:13:58.821671 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:58.820310    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 19 01:13:58.821671 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:58.821486    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 19 01:13:58.831476 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:58.831004    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"]
Sep 19 01:13:58.913709 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:58.913628    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8c057f6b-b862-40fa-b1dd-deb4c2102eed-secret-thanos-querier-tls\") pod \"thanos-querier-688f75fbf5-kvmhp\" (UID: \"8c057f6b-b862-40fa-b1dd-deb4c2102eed\") " pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:58.913709 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:58.913684    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8c057f6b-b862-40fa-b1dd-deb4c2102eed-secret-grpc-tls\") pod \"thanos-querier-688f75fbf5-kvmhp\" (UID: \"8c057f6b-b862-40fa-b1dd-deb4c2102eed\") " pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:58.913944 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:58.913718    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8c057f6b-b862-40fa-b1dd-deb4c2102eed-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-688f75fbf5-kvmhp\" (UID: \"8c057f6b-b862-40fa-b1dd-deb4c2102eed\") " pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:58.913944 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:58.913748    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8c057f6b-b862-40fa-b1dd-deb4c2102eed-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-688f75fbf5-kvmhp\" (UID: \"8c057f6b-b862-40fa-b1dd-deb4c2102eed\") " pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:58.913944 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:58.913774    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c057f6b-b862-40fa-b1dd-deb4c2102eed-metrics-client-ca\") pod \"thanos-querier-688f75fbf5-kvmhp\" (UID: \"8c057f6b-b862-40fa-b1dd-deb4c2102eed\") " pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:58.913944 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:58.913802    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5ck8\" (UniqueName: \"kubernetes.io/projected/8c057f6b-b862-40fa-b1dd-deb4c2102eed-kube-api-access-t5ck8\") pod \"thanos-querier-688f75fbf5-kvmhp\" (UID: \"8c057f6b-b862-40fa-b1dd-deb4c2102eed\") " pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:58.913944 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:58.913840    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8c057f6b-b862-40fa-b1dd-deb4c2102eed-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-688f75fbf5-kvmhp\" (UID: \"8c057f6b-b862-40fa-b1dd-deb4c2102eed\") " pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:58.913944 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:58.913875    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8c057f6b-b862-40fa-b1dd-deb4c2102eed-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-688f75fbf5-kvmhp\" (UID: \"8c057f6b-b862-40fa-b1dd-deb4c2102eed\") " pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:59.015207 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:59.015169    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8c057f6b-b862-40fa-b1dd-deb4c2102eed-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-688f75fbf5-kvmhp\" (UID: \"8c057f6b-b862-40fa-b1dd-deb4c2102eed\") " pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:59.015372 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:59.015224    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8c057f6b-b862-40fa-b1dd-deb4c2102eed-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-688f75fbf5-kvmhp\" (UID: \"8c057f6b-b862-40fa-b1dd-deb4c2102eed\") " pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:59.015372 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:59.015297    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8c057f6b-b862-40fa-b1dd-deb4c2102eed-secret-thanos-querier-tls\") pod \"thanos-querier-688f75fbf5-kvmhp\" (UID: \"8c057f6b-b862-40fa-b1dd-deb4c2102eed\") " pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:59.015372 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:59.015332    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8c057f6b-b862-40fa-b1dd-deb4c2102eed-secret-grpc-tls\") pod \"thanos-querier-688f75fbf5-kvmhp\" (UID: \"8c057f6b-b862-40fa-b1dd-deb4c2102eed\") " pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:59.015372 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:59.015366    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8c057f6b-b862-40fa-b1dd-deb4c2102eed-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-688f75fbf5-kvmhp\" (UID: \"8c057f6b-b862-40fa-b1dd-deb4c2102eed\") " pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:59.015610 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:59.015394    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8c057f6b-b862-40fa-b1dd-deb4c2102eed-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-688f75fbf5-kvmhp\" (UID: \"8c057f6b-b862-40fa-b1dd-deb4c2102eed\") " pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:59.015610 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:59.015545    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c057f6b-b862-40fa-b1dd-deb4c2102eed-metrics-client-ca\") pod \"thanos-querier-688f75fbf5-kvmhp\" (UID: \"8c057f6b-b862-40fa-b1dd-deb4c2102eed\") " pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:59.015610 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:59.015594    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5ck8\" (UniqueName: \"kubernetes.io/projected/8c057f6b-b862-40fa-b1dd-deb4c2102eed-kube-api-access-t5ck8\") pod \"thanos-querier-688f75fbf5-kvmhp\" (UID: \"8c057f6b-b862-40fa-b1dd-deb4c2102eed\") " pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:59.016407 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:59.016349    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c057f6b-b862-40fa-b1dd-deb4c2102eed-metrics-client-ca\") pod \"thanos-querier-688f75fbf5-kvmhp\" (UID: \"8c057f6b-b862-40fa-b1dd-deb4c2102eed\") " pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:59.018335 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:59.018311    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8c057f6b-b862-40fa-b1dd-deb4c2102eed-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-688f75fbf5-kvmhp\" (UID: \"8c057f6b-b862-40fa-b1dd-deb4c2102eed\") " pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:59.018462 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:59.018389    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8c057f6b-b862-40fa-b1dd-deb4c2102eed-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-688f75fbf5-kvmhp\" (UID: \"8c057f6b-b862-40fa-b1dd-deb4c2102eed\") " pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:59.018769 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:59.018747    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8c057f6b-b862-40fa-b1dd-deb4c2102eed-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-688f75fbf5-kvmhp\" (UID: \"8c057f6b-b862-40fa-b1dd-deb4c2102eed\") " pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:59.018974 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:59.018949    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8c057f6b-b862-40fa-b1dd-deb4c2102eed-secret-grpc-tls\") pod \"thanos-querier-688f75fbf5-kvmhp\" (UID: \"8c057f6b-b862-40fa-b1dd-deb4c2102eed\") " pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:59.019045 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:59.018991    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8c057f6b-b862-40fa-b1dd-deb4c2102eed-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-688f75fbf5-kvmhp\" (UID: \"8c057f6b-b862-40fa-b1dd-deb4c2102eed\") " pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:59.019045 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:59.019009    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8c057f6b-b862-40fa-b1dd-deb4c2102eed-secret-thanos-querier-tls\") pod \"thanos-querier-688f75fbf5-kvmhp\" (UID: \"8c057f6b-b862-40fa-b1dd-deb4c2102eed\") " pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:59.026552 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:59.026529    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5ck8\" (UniqueName: \"kubernetes.io/projected/8c057f6b-b862-40fa-b1dd-deb4c2102eed-kube-api-access-t5ck8\") pod \"thanos-querier-688f75fbf5-kvmhp\" (UID: \"8c057f6b-b862-40fa-b1dd-deb4c2102eed\") " pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:59.129090 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:59.129062    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:13:59.613684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:59.613652    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn" event={"ID":"551bcebb-cd3c-4ca5-ac56-54c6551248b0","Type":"ContainerStarted","Data":"7acce0afb0aa484fdcdfacb075a20605886ca5b49021d8d11e1e4bff1091c553"}
Sep 19 01:13:59.613810 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:59.613721    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn" event={"ID":"551bcebb-cd3c-4ca5-ac56-54c6551248b0","Type":"ContainerStarted","Data":"2a644e79ab548435828cea06e2b0da01fb6d228198e6adf52d0c62f3de5a403c"}
Sep 19 01:13:59.615341 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:59.615318    2580 generic.go:356] "Generic (PLEG): container finished" podID="e43af3ba-8b7a-488a-98e1-0acd8e823a1f" containerID="6d4eae171a5e88c652bc63e36db739a3f42ce858e68eb1eff1b40fd74e1722ea" exitCode=0
Sep 19 01:13:59.615456 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:59.615419    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8kpmp" event={"ID":"e43af3ba-8b7a-488a-98e1-0acd8e823a1f","Type":"ContainerDied","Data":"6d4eae171a5e88c652bc63e36db739a3f42ce858e68eb1eff1b40fd74e1722ea"}
Sep 19 01:13:59.618763 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:59.618732    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-86gqf" event={"ID":"31e2d5af-7ffd-4089-8628-eca0941659a9","Type":"ContainerStarted","Data":"6dc64e95c685af3c25da2fb21c59f5940fe3f1338cdf70ac37b43439cf450bc0"}
Sep 19 01:13:59.744397 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:59.744344    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-86gqf" podStartSLOduration=2.382786415 podStartE2EDuration="3.744319038s" totalImagesPullingTime="1.361532623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:13:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:13:59.742695274 +0000 UTC m=+94.649557335" watchObservedRunningTime="2026-09-19 01:13:59.744319038 +0000 UTC m=+94.651181087"
Sep 19 01:13:59.747615 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:13:59.747591    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"]
Sep 19 01:13:59.750477 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:13:59.750433    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c057f6b_b862_40fa_b1dd_deb4c2102eed.slice/crio-64b36e0a8306f94830fbd0aff4d93163f05ee2967505dbd74b5ddadbb4f32517 WatchSource:0}: Error finding container 64b36e0a8306f94830fbd0aff4d93163f05ee2967505dbd74b5ddadbb4f32517: Status 404 returned error can't find the container with id 64b36e0a8306f94830fbd0aff4d93163f05ee2967505dbd74b5ddadbb4f32517
Sep 19 01:14:00.628067 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:00.628029    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn" event={"ID":"551bcebb-cd3c-4ca5-ac56-54c6551248b0","Type":"ContainerStarted","Data":"e28aa58ea59b6b3249ed3b1e6166989b06f9519ede1b759422669a2e834e62ca"}
Sep 19 01:14:00.629479 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:00.629448    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp" event={"ID":"8c057f6b-b862-40fa-b1dd-deb4c2102eed","Type":"ContainerStarted","Data":"64b36e0a8306f94830fbd0aff4d93163f05ee2967505dbd74b5ddadbb4f32517"}
Sep 19 01:14:00.631849 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:00.631820    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8kpmp" event={"ID":"e43af3ba-8b7a-488a-98e1-0acd8e823a1f","Type":"ContainerStarted","Data":"0b9ca602da0296eb526a27c8bd0330d8f2aacdb76c6e5b10b9b6441282469722"}
Sep 19 01:14:00.631944 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:00.631863    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8kpmp" event={"ID":"e43af3ba-8b7a-488a-98e1-0acd8e823a1f","Type":"ContainerStarted","Data":"6b92cad6b4929a5a8b74482d64a66e9c2c82f92dabc9301183798c2f3ca98ff8"}
Sep 19 01:14:00.662874 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:00.662806    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-84c885658c-5q7kn" podStartSLOduration=3.169703948 podStartE2EDuration="4.662787603s" totalImagesPullingTime="1.493083655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:13:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:14:00.66027513 +0000 UTC m=+95.567137182" watchObservedRunningTime="2026-09-19 01:14:00.662787603 +0000 UTC m=+95.569649653"
Sep 19 01:14:00.687176 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:00.687128    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8kpmp" podStartSLOduration=3.511965125 podStartE2EDuration="4.687113858s" totalImagesPullingTime="1.098946614s" totalInitContainerRuntime="76.202119ms" isStatefulPod=true podCreationTimestamp="2026-09-19 01:13:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:14:00.687065569 +0000 UTC m=+95.593927643" watchObservedRunningTime="2026-09-19 01:14:00.687113858 +0000 UTC m=+95.593975905"
Sep 19 01:14:01.642114 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:01.642081    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp" event={"ID":"8c057f6b-b862-40fa-b1dd-deb4c2102eed","Type":"ContainerStarted","Data":"ac01914ff1fa3ccfb4e2f5d8b74ed5626fd5915ebe73adcba59685dae88b2c97"}
Sep 19 01:14:01.835051 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:01.834975    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-689cd6d4bf-rn5fq"]
Sep 19 01:14:01.838307 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:01.838285    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:01.849641 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:01.849614    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 19 01:14:01.849753 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:01.849620    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 19 01:14:01.849753 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:01.849658    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 19 01:14:01.849753 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:01.849679    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 19 01:14:01.850088 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:01.850070    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 19 01:14:01.851487 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:01.851473    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 19 01:14:01.852932 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:01.852919    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-22f44\""
Sep 19 01:14:01.853086 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:01.853076    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 19 01:14:01.858386 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:01.858361    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 19 01:14:01.864790 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:01.864769    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-689cd6d4bf-rn5fq"]
Sep 19 01:14:01.941649 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:01.941614    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/171bb032-bd91-468e-9309-d5c69fd65788-console-serving-cert\") pod \"console-689cd6d4bf-rn5fq\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") " pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:01.941809 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:01.941688    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/171bb032-bd91-468e-9309-d5c69fd65788-console-config\") pod \"console-689cd6d4bf-rn5fq\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") " pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:01.941809 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:01.941714    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/171bb032-bd91-468e-9309-d5c69fd65788-trusted-ca-bundle\") pod \"console-689cd6d4bf-rn5fq\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") " pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:01.941809 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:01.941740    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/171bb032-bd91-468e-9309-d5c69fd65788-oauth-serving-cert\") pod \"console-689cd6d4bf-rn5fq\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") " pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:01.941960 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:01.941814    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/171bb032-bd91-468e-9309-d5c69fd65788-service-ca\") pod \"console-689cd6d4bf-rn5fq\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") " pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:01.941960 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:01.941850    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/171bb032-bd91-468e-9309-d5c69fd65788-console-oauth-config\") pod \"console-689cd6d4bf-rn5fq\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") " pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:01.941960 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:01.941912    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2cf7\" (UniqueName: \"kubernetes.io/projected/171bb032-bd91-468e-9309-d5c69fd65788-kube-api-access-p2cf7\") pod \"console-689cd6d4bf-rn5fq\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") " pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:02.042921 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:02.042886    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/171bb032-bd91-468e-9309-d5c69fd65788-console-config\") pod \"console-689cd6d4bf-rn5fq\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") " pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:02.042921 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:02.042922    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/171bb032-bd91-468e-9309-d5c69fd65788-trusted-ca-bundle\") pod \"console-689cd6d4bf-rn5fq\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") " pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:02.043108 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:02.042943    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/171bb032-bd91-468e-9309-d5c69fd65788-oauth-serving-cert\") pod \"console-689cd6d4bf-rn5fq\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") " pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:02.043108 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:02.042990    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/171bb032-bd91-468e-9309-d5c69fd65788-service-ca\") pod \"console-689cd6d4bf-rn5fq\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") " pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:02.043108 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:02.043021    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/171bb032-bd91-468e-9309-d5c69fd65788-console-oauth-config\") pod \"console-689cd6d4bf-rn5fq\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") " pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:02.043108 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:02.043080    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2cf7\" (UniqueName: \"kubernetes.io/projected/171bb032-bd91-468e-9309-d5c69fd65788-kube-api-access-p2cf7\") pod \"console-689cd6d4bf-rn5fq\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") " pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:02.043288 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:02.043111    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/171bb032-bd91-468e-9309-d5c69fd65788-console-serving-cert\") pod \"console-689cd6d4bf-rn5fq\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") " pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:02.043873 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:02.043836    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/171bb032-bd91-468e-9309-d5c69fd65788-oauth-serving-cert\") pod \"console-689cd6d4bf-rn5fq\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") " pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:02.043873 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:02.043846    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/171bb032-bd91-468e-9309-d5c69fd65788-console-config\") pod \"console-689cd6d4bf-rn5fq\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") " pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:02.044046 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:02.043846    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/171bb032-bd91-468e-9309-d5c69fd65788-service-ca\") pod \"console-689cd6d4bf-rn5fq\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") " pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:02.044046 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:02.043925    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/171bb032-bd91-468e-9309-d5c69fd65788-trusted-ca-bundle\") pod \"console-689cd6d4bf-rn5fq\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") " pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:02.045613 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:02.045595    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/171bb032-bd91-468e-9309-d5c69fd65788-console-serving-cert\") pod \"console-689cd6d4bf-rn5fq\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") " pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:02.045670 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:02.045617    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/171bb032-bd91-468e-9309-d5c69fd65788-console-oauth-config\") pod \"console-689cd6d4bf-rn5fq\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") " pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:02.051569 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:02.051544    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2cf7\" (UniqueName: \"kubernetes.io/projected/171bb032-bd91-468e-9309-d5c69fd65788-kube-api-access-p2cf7\") pod \"console-689cd6d4bf-rn5fq\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") " pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:02.147404 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:02.147329    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:02.420287 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:02.420263    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-689cd6d4bf-rn5fq"]
Sep 19 01:14:02.422391 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:14:02.422367    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod171bb032_bd91_468e_9309_d5c69fd65788.slice/crio-d727161cfdc8551d20cf0762b811b228372a55dc9b975069686ac7ba57681c85 WatchSource:0}: Error finding container d727161cfdc8551d20cf0762b811b228372a55dc9b975069686ac7ba57681c85: Status 404 returned error can't find the container with id d727161cfdc8551d20cf0762b811b228372a55dc9b975069686ac7ba57681c85
Sep 19 01:14:02.648297 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:02.648242    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689cd6d4bf-rn5fq" event={"ID":"171bb032-bd91-468e-9309-d5c69fd65788","Type":"ContainerStarted","Data":"d727161cfdc8551d20cf0762b811b228372a55dc9b975069686ac7ba57681c85"}
Sep 19 01:14:02.653825 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:02.653033    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp" event={"ID":"8c057f6b-b862-40fa-b1dd-deb4c2102eed","Type":"ContainerStarted","Data":"72604cb6e9d94fe88f355e4e211dd107f334b6d92102df0f3c47e9c67d7e05ca"}
Sep 19 01:14:02.653825 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:02.653108    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp" event={"ID":"8c057f6b-b862-40fa-b1dd-deb4c2102eed","Type":"ContainerStarted","Data":"323ffe04bc5032deacd2c9ac62cd68fb7e3e8bf215eb5ef12b73598385d15f41"}
Sep 19 01:14:03.433757 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:03.433325    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"]
Sep 19 01:14:03.664110 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:03.664072    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp" event={"ID":"8c057f6b-b862-40fa-b1dd-deb4c2102eed","Type":"ContainerStarted","Data":"f3cb687a9a04dc41ac5afcbe16970885219173e0b812d65ba4945680c9f47006"}
Sep 19 01:14:03.664591 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:03.664166    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp" event={"ID":"8c057f6b-b862-40fa-b1dd-deb4c2102eed","Type":"ContainerStarted","Data":"9ec84940e718731b85729c79adb5e8132091901431c67ddf2aaa34bb2a9be64d"}
Sep 19 01:14:03.664591 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:03.664210    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp" event={"ID":"8c057f6b-b862-40fa-b1dd-deb4c2102eed","Type":"ContainerStarted","Data":"89b322fdd895125732cd6a3121f60787fdf810e41e8d12f02a07d42965c94115"}
Sep 19 01:14:03.664591 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:03.664335    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:14:03.716958 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:03.716605    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp" podStartSLOduration=2.90864522 podStartE2EDuration="5.716582498s" totalImagesPullingTime="2.807937278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:13:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:14:03.713113912 +0000 UTC m=+98.619975961" watchObservedRunningTime="2026-09-19 01:14:03.716582498 +0000 UTC m=+98.623444552"
Sep 19 01:14:06.680848 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:06.680804    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689cd6d4bf-rn5fq" event={"ID":"171bb032-bd91-468e-9309-d5c69fd65788","Type":"ContainerStarted","Data":"4fa1008408e69115b4704dbe76f5006cb0554045964b1766453fbe5f409048e5"}
Sep 19 01:14:06.736044 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:06.735994    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-689cd6d4bf-rn5fq" podStartSLOduration=1.579396494 podStartE2EDuration="5.735975271s" totalImagesPullingTime="4.156578777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:14:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:14:06.733338328 +0000 UTC m=+101.640200377" watchObservedRunningTime="2026-09-19 01:14:06.735975271 +0000 UTC m=+101.642837318"
Sep 19 01:14:09.009293 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:09.009252    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-689cd6d4bf-rn5fq"]
Sep 19 01:14:09.677136 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:09.677109    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-688f75fbf5-kvmhp"
Sep 19 01:14:12.147589 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:12.147558    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:28.463644 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.463591    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw" podUID="a439b5fa-a142-4815-b145-c3278eea25a0" containerName="registry" containerID="cri-o://5ed45ac5e3e58f6d83bbf8608c0b1e166132481fa608bb032128c8f1a430b9c0" gracePeriod=30
Sep 19 01:14:28.705717 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.705695    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:14:28.763385 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.763361    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a439b5fa-a142-4815-b145-c3278eea25a0-installation-pull-secrets\") pod \"a439b5fa-a142-4815-b145-c3278eea25a0\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") "
Sep 19 01:14:28.763559 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.763417    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a439b5fa-a142-4815-b145-c3278eea25a0-image-registry-private-configuration\") pod \"a439b5fa-a142-4815-b145-c3278eea25a0\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") "
Sep 19 01:14:28.763559 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.763451    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a439b5fa-a142-4815-b145-c3278eea25a0-trusted-ca\") pod \"a439b5fa-a142-4815-b145-c3278eea25a0\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") "
Sep 19 01:14:28.763559 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.763469    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-registry-tls\") pod \"a439b5fa-a142-4815-b145-c3278eea25a0\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") "
Sep 19 01:14:28.763559 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.763517    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fg64\" (UniqueName: \"kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-kube-api-access-6fg64\") pod \"a439b5fa-a142-4815-b145-c3278eea25a0\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") "
Sep 19 01:14:28.763559 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.763547    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a439b5fa-a142-4815-b145-c3278eea25a0-ca-trust-extracted\") pod \"a439b5fa-a142-4815-b145-c3278eea25a0\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") "
Sep 19 01:14:28.763868 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.763596    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a439b5fa-a142-4815-b145-c3278eea25a0-registry-certificates\") pod \"a439b5fa-a142-4815-b145-c3278eea25a0\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") "
Sep 19 01:14:28.763868 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.763619    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-bound-sa-token\") pod \"a439b5fa-a142-4815-b145-c3278eea25a0\" (UID: \"a439b5fa-a142-4815-b145-c3278eea25a0\") "
Sep 19 01:14:28.766377 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.764377    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a439b5fa-a142-4815-b145-c3278eea25a0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a439b5fa-a142-4815-b145-c3278eea25a0" (UID: "a439b5fa-a142-4815-b145-c3278eea25a0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:14:28.766377 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.764627    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a439b5fa-a142-4815-b145-c3278eea25a0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a439b5fa-a142-4815-b145-c3278eea25a0" (UID: "a439b5fa-a142-4815-b145-c3278eea25a0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:14:28.766978 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.766930    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a439b5fa-a142-4815-b145-c3278eea25a0" (UID: "a439b5fa-a142-4815-b145-c3278eea25a0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:14:28.767088 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.766942    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a439b5fa-a142-4815-b145-c3278eea25a0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a439b5fa-a142-4815-b145-c3278eea25a0" (UID: "a439b5fa-a142-4815-b145-c3278eea25a0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:14:28.767088 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.766984    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a439b5fa-a142-4815-b145-c3278eea25a0" (UID: "a439b5fa-a142-4815-b145-c3278eea25a0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:14:28.767088 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.767031    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a439b5fa-a142-4815-b145-c3278eea25a0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a439b5fa-a142-4815-b145-c3278eea25a0" (UID: "a439b5fa-a142-4815-b145-c3278eea25a0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:14:28.767221 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.767138    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-kube-api-access-6fg64" (OuterVolumeSpecName: "kube-api-access-6fg64") pod "a439b5fa-a142-4815-b145-c3278eea25a0" (UID: "a439b5fa-a142-4815-b145-c3278eea25a0"). InnerVolumeSpecName "kube-api-access-6fg64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:14:28.768984 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.768949    2580 generic.go:356] "Generic (PLEG): container finished" podID="a439b5fa-a142-4815-b145-c3278eea25a0" containerID="5ed45ac5e3e58f6d83bbf8608c0b1e166132481fa608bb032128c8f1a430b9c0" exitCode=0
Sep 19 01:14:28.769086 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.768994    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw" event={"ID":"a439b5fa-a142-4815-b145-c3278eea25a0","Type":"ContainerDied","Data":"5ed45ac5e3e58f6d83bbf8608c0b1e166132481fa608bb032128c8f1a430b9c0"}
Sep 19 01:14:28.769086 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.769047    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw" event={"ID":"a439b5fa-a142-4815-b145-c3278eea25a0","Type":"ContainerDied","Data":"8ed805986ed4f74963e2917a0acf88bb76c1778c28a0d6ba4e5cde1bce215048"}
Sep 19 01:14:28.769086 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.769053    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"
Sep 19 01:14:28.769086 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.769076    2580 scope.go:117] "RemoveContainer" containerID="5ed45ac5e3e58f6d83bbf8608c0b1e166132481fa608bb032128c8f1a430b9c0"
Sep 19 01:14:28.775432 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.775403    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a439b5fa-a142-4815-b145-c3278eea25a0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a439b5fa-a142-4815-b145-c3278eea25a0" (UID: "a439b5fa-a142-4815-b145-c3278eea25a0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:14:28.785117 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.785100    2580 scope.go:117] "RemoveContainer" containerID="5ed45ac5e3e58f6d83bbf8608c0b1e166132481fa608bb032128c8f1a430b9c0"
Sep 19 01:14:28.785422 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:14:28.785399    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ed45ac5e3e58f6d83bbf8608c0b1e166132481fa608bb032128c8f1a430b9c0\": container with ID starting with 5ed45ac5e3e58f6d83bbf8608c0b1e166132481fa608bb032128c8f1a430b9c0 not found: ID does not exist" containerID="5ed45ac5e3e58f6d83bbf8608c0b1e166132481fa608bb032128c8f1a430b9c0"
Sep 19 01:14:28.785472 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.785434    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ed45ac5e3e58f6d83bbf8608c0b1e166132481fa608bb032128c8f1a430b9c0"} err="failed to get container status \"5ed45ac5e3e58f6d83bbf8608c0b1e166132481fa608bb032128c8f1a430b9c0\": rpc error: code = NotFound desc = could not find container \"5ed45ac5e3e58f6d83bbf8608c0b1e166132481fa608bb032128c8f1a430b9c0\": container with ID starting with 5ed45ac5e3e58f6d83bbf8608c0b1e166132481fa608bb032128c8f1a430b9c0 not found: ID does not exist"
Sep 19 01:14:28.864453 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.864419    2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a439b5fa-a142-4815-b145-c3278eea25a0-installation-pull-secrets\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:14:28.864453 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.864448    2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a439b5fa-a142-4815-b145-c3278eea25a0-image-registry-private-configuration\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:14:28.864453 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.864462    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a439b5fa-a142-4815-b145-c3278eea25a0-trusted-ca\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:14:28.864695 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.864471    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-registry-tls\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:14:28.864695 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.864480    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fg64\" (UniqueName: \"kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-kube-api-access-6fg64\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:14:28.864695 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.864488    2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a439b5fa-a142-4815-b145-c3278eea25a0-ca-trust-extracted\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:14:28.864695 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.864519    2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a439b5fa-a142-4815-b145-c3278eea25a0-registry-certificates\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:14:28.864695 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:28.864527    2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a439b5fa-a142-4815-b145-c3278eea25a0-bound-sa-token\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:14:29.092890 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:29.092861    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"]
Sep 19 01:14:29.101794 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:29.101770    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5d7bd8c8fd-kxlmw"]
Sep 19 01:14:31.061966 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:31.061934    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a439b5fa-a142-4815-b145-c3278eea25a0" path="/var/lib/kubelet/pods/a439b5fa-a142-4815-b145-c3278eea25a0/volumes"
Sep 19 01:14:31.783888 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:31.783856    2580 generic.go:356] "Generic (PLEG): container finished" podID="2c41153e-4f57-4158-83b8-18563e2750e8" containerID="009a7a7acb73749cf1e2629a1021dfd162fc64382794e468553fcb93a5980f55" exitCode=0
Sep 19 01:14:31.784056 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:31.783934    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-bvc7f" event={"ID":"2c41153e-4f57-4158-83b8-18563e2750e8","Type":"ContainerDied","Data":"009a7a7acb73749cf1e2629a1021dfd162fc64382794e468553fcb93a5980f55"}
Sep 19 01:14:31.784263 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:31.784248    2580 scope.go:117] "RemoveContainer" containerID="009a7a7acb73749cf1e2629a1021dfd162fc64382794e468553fcb93a5980f55"
Sep 19 01:14:32.789886 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:32.789854    2580 generic.go:356] "Generic (PLEG): container finished" podID="8bdd86a3-9335-4e27-b183-c07df2869fb5" containerID="a2c0bc83f0a9ceb0662a5c08db5e3f38cae5e87ec696a4008ffca88254caec08" exitCode=0
Sep 19 01:14:32.790303 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:32.789936    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-44vdn" event={"ID":"8bdd86a3-9335-4e27-b183-c07df2869fb5","Type":"ContainerDied","Data":"a2c0bc83f0a9ceb0662a5c08db5e3f38cae5e87ec696a4008ffca88254caec08"}
Sep 19 01:14:32.790359 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:32.790309    2580 scope.go:117] "RemoveContainer" containerID="a2c0bc83f0a9ceb0662a5c08db5e3f38cae5e87ec696a4008ffca88254caec08"
Sep 19 01:14:32.791948 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:32.791928    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-bvc7f" event={"ID":"2c41153e-4f57-4158-83b8-18563e2750e8","Type":"ContainerStarted","Data":"af8a4f368dd815b615d8173920c7079b6edf5f2781d3359dc800a6992abc692a"}
Sep 19 01:14:33.799925 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:33.799894    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-44vdn" event={"ID":"8bdd86a3-9335-4e27-b183-c07df2869fb5","Type":"ContainerStarted","Data":"24235471389c4b495f44a4b177355eb32540573dc2162bf4d09d7729d7bdda50"}
Sep 19 01:14:34.028853 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.028817    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-689cd6d4bf-rn5fq" podUID="171bb032-bd91-468e-9309-d5c69fd65788" containerName="console" containerID="cri-o://4fa1008408e69115b4704dbe76f5006cb0554045964b1766453fbe5f409048e5" gracePeriod=15
Sep 19 01:14:34.316814 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.316785    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:34.414126 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.414052    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/171bb032-bd91-468e-9309-d5c69fd65788-oauth-serving-cert\") pod \"171bb032-bd91-468e-9309-d5c69fd65788\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") "
Sep 19 01:14:34.414126 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.414109    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/171bb032-bd91-468e-9309-d5c69fd65788-trusted-ca-bundle\") pod \"171bb032-bd91-468e-9309-d5c69fd65788\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") "
Sep 19 01:14:34.414126 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.414128    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2cf7\" (UniqueName: \"kubernetes.io/projected/171bb032-bd91-468e-9309-d5c69fd65788-kube-api-access-p2cf7\") pod \"171bb032-bd91-468e-9309-d5c69fd65788\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") "
Sep 19 01:14:34.414380 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.414150    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/171bb032-bd91-468e-9309-d5c69fd65788-console-config\") pod \"171bb032-bd91-468e-9309-d5c69fd65788\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") "
Sep 19 01:14:34.414380 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.414175    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/171bb032-bd91-468e-9309-d5c69fd65788-console-oauth-config\") pod \"171bb032-bd91-468e-9309-d5c69fd65788\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") "
Sep 19 01:14:34.414380 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.414215    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/171bb032-bd91-468e-9309-d5c69fd65788-console-serving-cert\") pod \"171bb032-bd91-468e-9309-d5c69fd65788\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") "
Sep 19 01:14:34.414380 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.414240    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/171bb032-bd91-468e-9309-d5c69fd65788-service-ca\") pod \"171bb032-bd91-468e-9309-d5c69fd65788\" (UID: \"171bb032-bd91-468e-9309-d5c69fd65788\") "
Sep 19 01:14:34.414618 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.414576    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/171bb032-bd91-468e-9309-d5c69fd65788-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "171bb032-bd91-468e-9309-d5c69fd65788" (UID: "171bb032-bd91-468e-9309-d5c69fd65788"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:14:34.414695 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.414669    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/171bb032-bd91-468e-9309-d5c69fd65788-console-config" (OuterVolumeSpecName: "console-config") pod "171bb032-bd91-468e-9309-d5c69fd65788" (UID: "171bb032-bd91-468e-9309-d5c69fd65788"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:14:34.414745 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.414685    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/171bb032-bd91-468e-9309-d5c69fd65788-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "171bb032-bd91-468e-9309-d5c69fd65788" (UID: "171bb032-bd91-468e-9309-d5c69fd65788"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:14:34.414819 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.414797    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/171bb032-bd91-468e-9309-d5c69fd65788-service-ca" (OuterVolumeSpecName: "service-ca") pod "171bb032-bd91-468e-9309-d5c69fd65788" (UID: "171bb032-bd91-468e-9309-d5c69fd65788"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:14:34.416469 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.416438    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/171bb032-bd91-468e-9309-d5c69fd65788-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "171bb032-bd91-468e-9309-d5c69fd65788" (UID: "171bb032-bd91-468e-9309-d5c69fd65788"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:14:34.416660 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.416471    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/171bb032-bd91-468e-9309-d5c69fd65788-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "171bb032-bd91-468e-9309-d5c69fd65788" (UID: "171bb032-bd91-468e-9309-d5c69fd65788"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:14:34.416660 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.416484    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/171bb032-bd91-468e-9309-d5c69fd65788-kube-api-access-p2cf7" (OuterVolumeSpecName: "kube-api-access-p2cf7") pod "171bb032-bd91-468e-9309-d5c69fd65788" (UID: "171bb032-bd91-468e-9309-d5c69fd65788"). InnerVolumeSpecName "kube-api-access-p2cf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:14:34.515690 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.515659    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/171bb032-bd91-468e-9309-d5c69fd65788-console-config\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:14:34.515690 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.515685    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/171bb032-bd91-468e-9309-d5c69fd65788-console-oauth-config\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:14:34.515887 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.515699    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/171bb032-bd91-468e-9309-d5c69fd65788-console-serving-cert\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:14:34.515887 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.515711    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/171bb032-bd91-468e-9309-d5c69fd65788-service-ca\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:14:34.515887 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.515722    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/171bb032-bd91-468e-9309-d5c69fd65788-oauth-serving-cert\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:14:34.516005 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.515733    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/171bb032-bd91-468e-9309-d5c69fd65788-trusted-ca-bundle\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:14:34.516005 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.515910    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2cf7\" (UniqueName: \"kubernetes.io/projected/171bb032-bd91-468e-9309-d5c69fd65788-kube-api-access-p2cf7\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:14:34.805238 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.805199    2580 generic.go:356] "Generic (PLEG): container finished" podID="171bb032-bd91-468e-9309-d5c69fd65788" containerID="4fa1008408e69115b4704dbe76f5006cb0554045964b1766453fbe5f409048e5" exitCode=0
Sep 19 01:14:34.805710 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.805259    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689cd6d4bf-rn5fq"
Sep 19 01:14:34.805710 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.805285    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689cd6d4bf-rn5fq" event={"ID":"171bb032-bd91-468e-9309-d5c69fd65788","Type":"ContainerDied","Data":"4fa1008408e69115b4704dbe76f5006cb0554045964b1766453fbe5f409048e5"}
Sep 19 01:14:34.805710 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.805358    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689cd6d4bf-rn5fq" event={"ID":"171bb032-bd91-468e-9309-d5c69fd65788","Type":"ContainerDied","Data":"d727161cfdc8551d20cf0762b811b228372a55dc9b975069686ac7ba57681c85"}
Sep 19 01:14:34.805710 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.805377    2580 scope.go:117] "RemoveContainer" containerID="4fa1008408e69115b4704dbe76f5006cb0554045964b1766453fbe5f409048e5"
Sep 19 01:14:34.814570 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.814553    2580 scope.go:117] "RemoveContainer" containerID="4fa1008408e69115b4704dbe76f5006cb0554045964b1766453fbe5f409048e5"
Sep 19 01:14:34.814831 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:14:34.814813    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fa1008408e69115b4704dbe76f5006cb0554045964b1766453fbe5f409048e5\": container with ID starting with 4fa1008408e69115b4704dbe76f5006cb0554045964b1766453fbe5f409048e5 not found: ID does not exist" containerID="4fa1008408e69115b4704dbe76f5006cb0554045964b1766453fbe5f409048e5"
Sep 19 01:14:34.814876 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.814838    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fa1008408e69115b4704dbe76f5006cb0554045964b1766453fbe5f409048e5"} err="failed to get container status \"4fa1008408e69115b4704dbe76f5006cb0554045964b1766453fbe5f409048e5\": rpc error: code = NotFound desc = could not find container \"4fa1008408e69115b4704dbe76f5006cb0554045964b1766453fbe5f409048e5\": container with ID starting with 4fa1008408e69115b4704dbe76f5006cb0554045964b1766453fbe5f409048e5 not found: ID does not exist"
Sep 19 01:14:34.828733 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.828709    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-689cd6d4bf-rn5fq"]
Sep 19 01:14:34.834419 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:34.834397    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-689cd6d4bf-rn5fq"]
Sep 19 01:14:35.062659 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:35.062580    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="171bb032-bd91-468e-9309-d5c69fd65788" path="/var/lib/kubelet/pods/171bb032-bd91-468e-9309-d5c69fd65788/volumes"
Sep 19 01:14:41.841265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:41.841234    2580 generic.go:356] "Generic (PLEG): container finished" podID="39355481-6d16-4553-961a-f03ca171e712" containerID="3d1b365a4af2231417d9b0ef5255e38e738299f57134a96a4945752ad4fb5889" exitCode=0
Sep 19 01:14:41.841777 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:41.841271    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m" event={"ID":"39355481-6d16-4553-961a-f03ca171e712","Type":"ContainerDied","Data":"3d1b365a4af2231417d9b0ef5255e38e738299f57134a96a4945752ad4fb5889"}
Sep 19 01:14:41.841777 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:41.841729    2580 scope.go:117] "RemoveContainer" containerID="3d1b365a4af2231417d9b0ef5255e38e738299f57134a96a4945752ad4fb5889"
Sep 19 01:14:42.847340 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:14:42.847303    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-p6l4m" event={"ID":"39355481-6d16-4553-961a-f03ca171e712","Type":"ContainerStarted","Data":"431dc0c1a46f1ca1315772b8d49f8f2d6521f94b2b8dbaac9beddc3e238865f8"}
Sep 19 01:15:14.992519 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:14.992426    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bfb8b856-s8hkr"]
Sep 19 01:15:14.993206 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:14.992768    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a439b5fa-a142-4815-b145-c3278eea25a0" containerName="registry"
Sep 19 01:15:14.993206 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:14.992780    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a439b5fa-a142-4815-b145-c3278eea25a0" containerName="registry"
Sep 19 01:15:14.993206 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:14.992788    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="171bb032-bd91-468e-9309-d5c69fd65788" containerName="console"
Sep 19 01:15:14.993206 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:14.992794    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="171bb032-bd91-468e-9309-d5c69fd65788" containerName="console"
Sep 19 01:15:14.993206 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:14.992839    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="171bb032-bd91-468e-9309-d5c69fd65788" containerName="console"
Sep 19 01:15:14.993206 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:14.992846    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a439b5fa-a142-4815-b145-c3278eea25a0" containerName="registry"
Sep 19 01:15:14.995410 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:14.995392    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:14.999408 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:14.999386    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 19 01:15:14.999517 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:14.999389    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 19 01:15:14.999932 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:14.999912    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-22f44\""
Sep 19 01:15:15.000046 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:14.999917    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 19 01:15:15.000258 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.000235    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 19 01:15:15.000258 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.000245    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 19 01:15:15.000435 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.000325    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 19 01:15:15.000478 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.000441    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 19 01:15:15.005175 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.005154    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 19 01:15:15.014507 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.014477    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bfb8b856-s8hkr"]
Sep 19 01:15:15.139808 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.139781    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-oauth-serving-cert\") pod \"console-bfb8b856-s8hkr\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") " pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:15.139978 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.139811    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-console-oauth-config\") pod \"console-bfb8b856-s8hkr\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") " pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:15.139978 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.139838    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nxwp\" (UniqueName: \"kubernetes.io/projected/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-kube-api-access-6nxwp\") pod \"console-bfb8b856-s8hkr\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") " pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:15.139978 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.139856    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-console-config\") pod \"console-bfb8b856-s8hkr\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") " pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:15.139978 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.139903    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-service-ca\") pod \"console-bfb8b856-s8hkr\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") " pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:15.139978 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.139945    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-console-serving-cert\") pod \"console-bfb8b856-s8hkr\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") " pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:15.139978 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.139970    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-trusted-ca-bundle\") pod \"console-bfb8b856-s8hkr\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") " pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:15.240732 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.240701    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-console-serving-cert\") pod \"console-bfb8b856-s8hkr\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") " pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:15.240732 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.240732    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-trusted-ca-bundle\") pod \"console-bfb8b856-s8hkr\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") " pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:15.240961 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.240823    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-oauth-serving-cert\") pod \"console-bfb8b856-s8hkr\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") " pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:15.240961 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.240845    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-console-oauth-config\") pod \"console-bfb8b856-s8hkr\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") " pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:15.240961 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.240880    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nxwp\" (UniqueName: \"kubernetes.io/projected/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-kube-api-access-6nxwp\") pod \"console-bfb8b856-s8hkr\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") " pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:15.240961 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.240910    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-console-config\") pod \"console-bfb8b856-s8hkr\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") " pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:15.240961 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.240933    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-service-ca\") pod \"console-bfb8b856-s8hkr\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") " pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:15.241627 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.241604    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-oauth-serving-cert\") pod \"console-bfb8b856-s8hkr\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") " pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:15.241738 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.241630    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-console-config\") pod \"console-bfb8b856-s8hkr\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") " pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:15.241738 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.241725    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-service-ca\") pod \"console-bfb8b856-s8hkr\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") " pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:15.241948 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.241926    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-trusted-ca-bundle\") pod \"console-bfb8b856-s8hkr\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") " pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:15.243444 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.243389    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-console-oauth-config\") pod \"console-bfb8b856-s8hkr\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") " pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:15.243587 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.243468    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-console-serving-cert\") pod \"console-bfb8b856-s8hkr\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") " pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:15.248851 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.248832    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nxwp\" (UniqueName: \"kubernetes.io/projected/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-kube-api-access-6nxwp\") pod \"console-bfb8b856-s8hkr\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") " pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:15.305947 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.305914    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:15.580576 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.580389    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bfb8b856-s8hkr"]
Sep 19 01:15:15.583279 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:15:15.583247    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdaf3f4e_5b9f_43a5_898d_a27646a0a9df.slice/crio-4f6949f0c3bf5c8e307e006fc6708af5ac3d8f8c026cb66f7f2373262928b100 WatchSource:0}: Error finding container 4f6949f0c3bf5c8e307e006fc6708af5ac3d8f8c026cb66f7f2373262928b100: Status 404 returned error can't find the container with id 4f6949f0c3bf5c8e307e006fc6708af5ac3d8f8c026cb66f7f2373262928b100
Sep 19 01:15:15.997678 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.997642    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bfb8b856-s8hkr" event={"ID":"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df","Type":"ContainerStarted","Data":"50115a5f4296ffdeafa5b71bffe4e0c755b774e2787ed7f218a7c567658a037a"}
Sep 19 01:15:15.998061 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:15.997716    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bfb8b856-s8hkr" event={"ID":"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df","Type":"ContainerStarted","Data":"4f6949f0c3bf5c8e307e006fc6708af5ac3d8f8c026cb66f7f2373262928b100"}
Sep 19 01:15:16.016323 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:16.016272    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bfb8b856-s8hkr" podStartSLOduration=2.016253588 podStartE2EDuration="2.016253588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:15:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:15:16.015398072 +0000 UTC m=+170.922260120" watchObservedRunningTime="2026-09-19 01:15:16.016253588 +0000 UTC m=+170.923115695"
Sep 19 01:15:25.307027 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:25.306988    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:25.307481 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:25.307072    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:25.311639 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:25.311620    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:15:26.041049 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:15:26.041020    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:16:05.486214 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:05.486179    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g"]
Sep 19 01:16:05.489781 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:05.489765    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g"
Sep 19 01:16:05.492141 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:05.492116    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 19 01:16:05.492141 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:05.492116    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kfj97\""
Sep 19 01:16:05.492671 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:05.492658    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 19 01:16:05.504183 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:05.504157    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g"]
Sep 19 01:16:05.654611 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:05.654573    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5cns\" (UniqueName: \"kubernetes.io/projected/62164f1d-fb78-4b5c-8adb-bb0e47d6da62-kube-api-access-d5cns\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g\" (UID: \"62164f1d-fb78-4b5c-8adb-bb0e47d6da62\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g"
Sep 19 01:16:05.654805 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:05.654626    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/62164f1d-fb78-4b5c-8adb-bb0e47d6da62-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g\" (UID: \"62164f1d-fb78-4b5c-8adb-bb0e47d6da62\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g"
Sep 19 01:16:05.654805 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:05.654677    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/62164f1d-fb78-4b5c-8adb-bb0e47d6da62-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g\" (UID: \"62164f1d-fb78-4b5c-8adb-bb0e47d6da62\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g"
Sep 19 01:16:05.755938 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:05.755860    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5cns\" (UniqueName: \"kubernetes.io/projected/62164f1d-fb78-4b5c-8adb-bb0e47d6da62-kube-api-access-d5cns\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g\" (UID: \"62164f1d-fb78-4b5c-8adb-bb0e47d6da62\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g"
Sep 19 01:16:05.755938 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:05.755909    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/62164f1d-fb78-4b5c-8adb-bb0e47d6da62-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g\" (UID: \"62164f1d-fb78-4b5c-8adb-bb0e47d6da62\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g"
Sep 19 01:16:05.755938 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:05.755934    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/62164f1d-fb78-4b5c-8adb-bb0e47d6da62-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g\" (UID: \"62164f1d-fb78-4b5c-8adb-bb0e47d6da62\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g"
Sep 19 01:16:05.756299 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:05.756277    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/62164f1d-fb78-4b5c-8adb-bb0e47d6da62-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g\" (UID: \"62164f1d-fb78-4b5c-8adb-bb0e47d6da62\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g"
Sep 19 01:16:05.756299 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:05.756291    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/62164f1d-fb78-4b5c-8adb-bb0e47d6da62-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g\" (UID: \"62164f1d-fb78-4b5c-8adb-bb0e47d6da62\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g"
Sep 19 01:16:05.764626 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:05.764599    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5cns\" (UniqueName: \"kubernetes.io/projected/62164f1d-fb78-4b5c-8adb-bb0e47d6da62-kube-api-access-d5cns\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g\" (UID: \"62164f1d-fb78-4b5c-8adb-bb0e47d6da62\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g"
Sep 19 01:16:05.798596 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:05.798571    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g"
Sep 19 01:16:06.074191 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:06.074122    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g"]
Sep 19 01:16:06.077397 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:16:06.077358    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62164f1d_fb78_4b5c_8adb_bb0e47d6da62.slice/crio-e4351288ce72678785b92ee0ee1d58946c1a69cd4a3a35f7f47657dbe18293a9 WatchSource:0}: Error finding container e4351288ce72678785b92ee0ee1d58946c1a69cd4a3a35f7f47657dbe18293a9: Status 404 returned error can't find the container with id e4351288ce72678785b92ee0ee1d58946c1a69cd4a3a35f7f47657dbe18293a9
Sep 19 01:16:06.200211 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:06.200176    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g" event={"ID":"62164f1d-fb78-4b5c-8adb-bb0e47d6da62","Type":"ContainerStarted","Data":"e4351288ce72678785b92ee0ee1d58946c1a69cd4a3a35f7f47657dbe18293a9"}
Sep 19 01:16:12.229583 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:12.229485    2580 generic.go:356] "Generic (PLEG): container finished" podID="62164f1d-fb78-4b5c-8adb-bb0e47d6da62" containerID="629b36c987cde0082e17a31ee21648918753d352abe43b09e12eeeb176b4445a" exitCode=0
Sep 19 01:16:12.229583 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:12.229542    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g" event={"ID":"62164f1d-fb78-4b5c-8adb-bb0e47d6da62","Type":"ContainerDied","Data":"629b36c987cde0082e17a31ee21648918753d352abe43b09e12eeeb176b4445a"}
Sep 19 01:16:15.244317 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:15.244283    2580 generic.go:356] "Generic (PLEG): container finished" podID="62164f1d-fb78-4b5c-8adb-bb0e47d6da62" containerID="19ef7da29fc712589926f359ff555fe7d501a28d29a9420d220b01299339a54e" exitCode=0
Sep 19 01:16:15.244733 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:15.244328    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g" event={"ID":"62164f1d-fb78-4b5c-8adb-bb0e47d6da62","Type":"ContainerDied","Data":"19ef7da29fc712589926f359ff555fe7d501a28d29a9420d220b01299339a54e"}
Sep 19 01:16:22.279077 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:22.279046    2580 generic.go:356] "Generic (PLEG): container finished" podID="62164f1d-fb78-4b5c-8adb-bb0e47d6da62" containerID="56c2f501d6fe926570c13d67dccb4e4ac6e7ef4758bb2843aec447913088303e" exitCode=0
Sep 19 01:16:22.279455 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:22.279101    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g" event={"ID":"62164f1d-fb78-4b5c-8adb-bb0e47d6da62","Type":"ContainerDied","Data":"56c2f501d6fe926570c13d67dccb4e4ac6e7ef4758bb2843aec447913088303e"}
Sep 19 01:16:23.405621 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:23.405596    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g"
Sep 19 01:16:23.499686 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:23.499650    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/62164f1d-fb78-4b5c-8adb-bb0e47d6da62-util\") pod \"62164f1d-fb78-4b5c-8adb-bb0e47d6da62\" (UID: \"62164f1d-fb78-4b5c-8adb-bb0e47d6da62\") "
Sep 19 01:16:23.499686 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:23.499688    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5cns\" (UniqueName: \"kubernetes.io/projected/62164f1d-fb78-4b5c-8adb-bb0e47d6da62-kube-api-access-d5cns\") pod \"62164f1d-fb78-4b5c-8adb-bb0e47d6da62\" (UID: \"62164f1d-fb78-4b5c-8adb-bb0e47d6da62\") "
Sep 19 01:16:23.499916 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:23.499725    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/62164f1d-fb78-4b5c-8adb-bb0e47d6da62-bundle\") pod \"62164f1d-fb78-4b5c-8adb-bb0e47d6da62\" (UID: \"62164f1d-fb78-4b5c-8adb-bb0e47d6da62\") "
Sep 19 01:16:23.500335 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:23.500308    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62164f1d-fb78-4b5c-8adb-bb0e47d6da62-bundle" (OuterVolumeSpecName: "bundle") pod "62164f1d-fb78-4b5c-8adb-bb0e47d6da62" (UID: "62164f1d-fb78-4b5c-8adb-bb0e47d6da62"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:16:23.501858 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:23.501838    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62164f1d-fb78-4b5c-8adb-bb0e47d6da62-kube-api-access-d5cns" (OuterVolumeSpecName: "kube-api-access-d5cns") pod "62164f1d-fb78-4b5c-8adb-bb0e47d6da62" (UID: "62164f1d-fb78-4b5c-8adb-bb0e47d6da62"). InnerVolumeSpecName "kube-api-access-d5cns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:16:23.503732 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:23.503705    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62164f1d-fb78-4b5c-8adb-bb0e47d6da62-util" (OuterVolumeSpecName: "util") pod "62164f1d-fb78-4b5c-8adb-bb0e47d6da62" (UID: "62164f1d-fb78-4b5c-8adb-bb0e47d6da62"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:16:23.600347 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:23.600286    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/62164f1d-fb78-4b5c-8adb-bb0e47d6da62-bundle\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:16:23.600347 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:23.600307    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/62164f1d-fb78-4b5c-8adb-bb0e47d6da62-util\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:16:23.600347 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:23.600316    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5cns\" (UniqueName: \"kubernetes.io/projected/62164f1d-fb78-4b5c-8adb-bb0e47d6da62-kube-api-access-d5cns\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:16:24.289085 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:24.289045    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g" event={"ID":"62164f1d-fb78-4b5c-8adb-bb0e47d6da62","Type":"ContainerDied","Data":"e4351288ce72678785b92ee0ee1d58946c1a69cd4a3a35f7f47657dbe18293a9"}
Sep 19 01:16:24.289251 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:24.289105    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4351288ce72678785b92ee0ee1d58946c1a69cd4a3a35f7f47657dbe18293a9"
Sep 19 01:16:24.289251 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:24.289140    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bzp8g"
Sep 19 01:16:28.731363 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:28.731329    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mtj9"]
Sep 19 01:16:28.731816 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:28.731697    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62164f1d-fb78-4b5c-8adb-bb0e47d6da62" containerName="util"
Sep 19 01:16:28.731816 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:28.731714    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="62164f1d-fb78-4b5c-8adb-bb0e47d6da62" containerName="util"
Sep 19 01:16:28.731816 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:28.731730    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62164f1d-fb78-4b5c-8adb-bb0e47d6da62" containerName="pull"
Sep 19 01:16:28.731816 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:28.731736    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="62164f1d-fb78-4b5c-8adb-bb0e47d6da62" containerName="pull"
Sep 19 01:16:28.731816 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:28.731744    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62164f1d-fb78-4b5c-8adb-bb0e47d6da62" containerName="extract"
Sep 19 01:16:28.731816 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:28.731749    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="62164f1d-fb78-4b5c-8adb-bb0e47d6da62" containerName="extract"
Sep 19 01:16:28.731816 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:28.731809    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="62164f1d-fb78-4b5c-8adb-bb0e47d6da62" containerName="extract"
Sep 19 01:16:28.734894 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:28.734878    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mtj9"
Sep 19 01:16:28.737943 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:28.737919    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-7lrhg\""
Sep 19 01:16:28.738046 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:28.738019    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 19 01:16:28.741567 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:28.741445    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 19 01:16:28.760867 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:28.760847    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mtj9"]
Sep 19 01:16:28.840060 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:28.840031    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a4f0920-150b-4440-acfc-f48eab734794-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7mtj9\" (UID: \"6a4f0920-150b-4440-acfc-f48eab734794\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mtj9"
Sep 19 01:16:28.840189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:28.840076    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwn9r\" (UniqueName: \"kubernetes.io/projected/6a4f0920-150b-4440-acfc-f48eab734794-kube-api-access-mwn9r\") pod \"cert-manager-operator-controller-manager-55c444bf86-7mtj9\" (UID: \"6a4f0920-150b-4440-acfc-f48eab734794\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mtj9"
Sep 19 01:16:28.940687 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:28.940644    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a4f0920-150b-4440-acfc-f48eab734794-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7mtj9\" (UID: \"6a4f0920-150b-4440-acfc-f48eab734794\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mtj9"
Sep 19 01:16:28.940832 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:28.940706    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwn9r\" (UniqueName: \"kubernetes.io/projected/6a4f0920-150b-4440-acfc-f48eab734794-kube-api-access-mwn9r\") pod \"cert-manager-operator-controller-manager-55c444bf86-7mtj9\" (UID: \"6a4f0920-150b-4440-acfc-f48eab734794\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mtj9"
Sep 19 01:16:28.941032 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:28.941012    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a4f0920-150b-4440-acfc-f48eab734794-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7mtj9\" (UID: \"6a4f0920-150b-4440-acfc-f48eab734794\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mtj9"
Sep 19 01:16:28.949387 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:28.949357    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwn9r\" (UniqueName: \"kubernetes.io/projected/6a4f0920-150b-4440-acfc-f48eab734794-kube-api-access-mwn9r\") pod \"cert-manager-operator-controller-manager-55c444bf86-7mtj9\" (UID: \"6a4f0920-150b-4440-acfc-f48eab734794\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mtj9"
Sep 19 01:16:29.043409 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:29.043378    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mtj9"
Sep 19 01:16:29.325099 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:29.325073    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mtj9"]
Sep 19 01:16:29.327547 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:16:29.327520    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a4f0920_150b_4440_acfc_f48eab734794.slice/crio-bd7e93af3ad63c7bfdd51e73b3fb0eee1b83ace483e53eabf92867ed4ee3efbb WatchSource:0}: Error finding container bd7e93af3ad63c7bfdd51e73b3fb0eee1b83ace483e53eabf92867ed4ee3efbb: Status 404 returned error can't find the container with id bd7e93af3ad63c7bfdd51e73b3fb0eee1b83ace483e53eabf92867ed4ee3efbb
Sep 19 01:16:30.315828 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:30.315794    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mtj9" event={"ID":"6a4f0920-150b-4440-acfc-f48eab734794","Type":"ContainerStarted","Data":"bd7e93af3ad63c7bfdd51e73b3fb0eee1b83ace483e53eabf92867ed4ee3efbb"}
Sep 19 01:16:35.340435 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:35.340401    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mtj9" event={"ID":"6a4f0920-150b-4440-acfc-f48eab734794","Type":"ContainerStarted","Data":"73402d3ab4a57fdb26f3ba2fa3af627375fef3bdbc388e3258672ed83dde6c7e"}
Sep 19 01:16:35.369703 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:35.369652    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mtj9" podStartSLOduration=2.065251095 podStartE2EDuration="7.369635791s" totalImagesPullingTime="5.304384696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:16:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:16:35.368001716 +0000 UTC m=+250.274863764" watchObservedRunningTime="2026-09-19 01:16:35.369635791 +0000 UTC m=+250.276497839"
Sep 19 01:16:37.478826 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:37.478785    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c"]
Sep 19 01:16:37.482563 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:37.482541    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c"
Sep 19 01:16:37.486797 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:37.486780    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 19 01:16:37.486797 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:37.486790    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 19 01:16:37.486957 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:37.486850    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kfj97\""
Sep 19 01:16:37.495696 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:37.495673    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c"]
Sep 19 01:16:37.612035 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:37.611997    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d1f64b3-110a-4a4d-9c40-62c20db48e36-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c\" (UID: \"9d1f64b3-110a-4a4d-9c40-62c20db48e36\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c"
Sep 19 01:16:37.612225 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:37.612038    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwpkz\" (UniqueName: \"kubernetes.io/projected/9d1f64b3-110a-4a4d-9c40-62c20db48e36-kube-api-access-pwpkz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c\" (UID: \"9d1f64b3-110a-4a4d-9c40-62c20db48e36\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c"
Sep 19 01:16:37.612225 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:37.612183    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d1f64b3-110a-4a4d-9c40-62c20db48e36-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c\" (UID: \"9d1f64b3-110a-4a4d-9c40-62c20db48e36\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c"
Sep 19 01:16:37.713004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:37.712978    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d1f64b3-110a-4a4d-9c40-62c20db48e36-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c\" (UID: \"9d1f64b3-110a-4a4d-9c40-62c20db48e36\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c"
Sep 19 01:16:37.713190 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:37.713014    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwpkz\" (UniqueName: \"kubernetes.io/projected/9d1f64b3-110a-4a4d-9c40-62c20db48e36-kube-api-access-pwpkz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c\" (UID: \"9d1f64b3-110a-4a4d-9c40-62c20db48e36\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c"
Sep 19 01:16:37.713190 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:37.713112    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d1f64b3-110a-4a4d-9c40-62c20db48e36-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c\" (UID: \"9d1f64b3-110a-4a4d-9c40-62c20db48e36\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c"
Sep 19 01:16:37.713358 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:37.713337    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d1f64b3-110a-4a4d-9c40-62c20db48e36-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c\" (UID: \"9d1f64b3-110a-4a4d-9c40-62c20db48e36\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c"
Sep 19 01:16:37.713431 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:37.713414    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d1f64b3-110a-4a4d-9c40-62c20db48e36-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c\" (UID: \"9d1f64b3-110a-4a4d-9c40-62c20db48e36\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c"
Sep 19 01:16:37.725409 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:37.725387    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwpkz\" (UniqueName: \"kubernetes.io/projected/9d1f64b3-110a-4a4d-9c40-62c20db48e36-kube-api-access-pwpkz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c\" (UID: \"9d1f64b3-110a-4a4d-9c40-62c20db48e36\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c"
Sep 19 01:16:37.791288 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:37.791261    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c"
Sep 19 01:16:38.061572 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:38.061467    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c"]
Sep 19 01:16:38.065062 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:16:38.065029    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d1f64b3_110a_4a4d_9c40_62c20db48e36.slice/crio-d6919455fa24f8c23ecce92be8c3878126c5f16d02c5bca99c0e5886706211ec WatchSource:0}: Error finding container d6919455fa24f8c23ecce92be8c3878126c5f16d02c5bca99c0e5886706211ec: Status 404 returned error can't find the container with id d6919455fa24f8c23ecce92be8c3878126c5f16d02c5bca99c0e5886706211ec
Sep 19 01:16:38.282158 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:38.282127    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zzchf"]
Sep 19 01:16:38.285389 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:38.285367    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-zzchf"
Sep 19 01:16:38.287455 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:38.287432    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 19 01:16:38.287579 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:38.287544    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-8zrvw\""
Sep 19 01:16:38.287579 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:38.287559    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 19 01:16:38.293442 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:38.293422    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zzchf"]
Sep 19 01:16:38.318493 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:38.318435    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxj8z\" (UniqueName: \"kubernetes.io/projected/da9e4c81-6d57-4cfb-8298-433cb0339a0c-kube-api-access-nxj8z\") pod \"cert-manager-webhook-6486ffd997-zzchf\" (UID: \"da9e4c81-6d57-4cfb-8298-433cb0339a0c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zzchf"
Sep 19 01:16:38.318493 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:38.318481    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da9e4c81-6d57-4cfb-8298-433cb0339a0c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zzchf\" (UID: \"da9e4c81-6d57-4cfb-8298-433cb0339a0c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zzchf"
Sep 19 01:16:38.354178 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:38.354155    2580 generic.go:356] "Generic (PLEG): container finished" podID="9d1f64b3-110a-4a4d-9c40-62c20db48e36" containerID="0d25115a6561e5aefac8b1778f6663f047efbbfe8ff2c8d675b4f0301b7b8325" exitCode=0
Sep 19 01:16:38.354272 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:38.354222    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c" event={"ID":"9d1f64b3-110a-4a4d-9c40-62c20db48e36","Type":"ContainerDied","Data":"0d25115a6561e5aefac8b1778f6663f047efbbfe8ff2c8d675b4f0301b7b8325"}
Sep 19 01:16:38.354312 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:38.354271    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c" event={"ID":"9d1f64b3-110a-4a4d-9c40-62c20db48e36","Type":"ContainerStarted","Data":"d6919455fa24f8c23ecce92be8c3878126c5f16d02c5bca99c0e5886706211ec"}
Sep 19 01:16:38.419549 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:38.419522    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxj8z\" (UniqueName: \"kubernetes.io/projected/da9e4c81-6d57-4cfb-8298-433cb0339a0c-kube-api-access-nxj8z\") pod \"cert-manager-webhook-6486ffd997-zzchf\" (UID: \"da9e4c81-6d57-4cfb-8298-433cb0339a0c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zzchf"
Sep 19 01:16:38.419694 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:38.419612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da9e4c81-6d57-4cfb-8298-433cb0339a0c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zzchf\" (UID: \"da9e4c81-6d57-4cfb-8298-433cb0339a0c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zzchf"
Sep 19 01:16:38.427964 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:38.427939    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da9e4c81-6d57-4cfb-8298-433cb0339a0c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zzchf\" (UID: \"da9e4c81-6d57-4cfb-8298-433cb0339a0c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zzchf"
Sep 19 01:16:38.428124 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:38.428107    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxj8z\" (UniqueName: \"kubernetes.io/projected/da9e4c81-6d57-4cfb-8298-433cb0339a0c-kube-api-access-nxj8z\") pod \"cert-manager-webhook-6486ffd997-zzchf\" (UID: \"da9e4c81-6d57-4cfb-8298-433cb0339a0c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zzchf"
Sep 19 01:16:38.619334 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:38.619246    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-zzchf"
Sep 19 01:16:38.895299 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:38.895227    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zzchf"]
Sep 19 01:16:38.898466 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:16:38.898437    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda9e4c81_6d57_4cfb_8298_433cb0339a0c.slice/crio-9d23028ee6c7234cd1c1429fb57f660dba6adc51abaa6ac358f05be5c31b9d5b WatchSource:0}: Error finding container 9d23028ee6c7234cd1c1429fb57f660dba6adc51abaa6ac358f05be5c31b9d5b: Status 404 returned error can't find the container with id 9d23028ee6c7234cd1c1429fb57f660dba6adc51abaa6ac358f05be5c31b9d5b
Sep 19 01:16:39.361415 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:39.361375    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-zzchf" event={"ID":"da9e4c81-6d57-4cfb-8298-433cb0339a0c","Type":"ContainerStarted","Data":"9d23028ee6c7234cd1c1429fb57f660dba6adc51abaa6ac358f05be5c31b9d5b"}
Sep 19 01:16:41.374003 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:41.373970    2580 generic.go:356] "Generic (PLEG): container finished" podID="9d1f64b3-110a-4a4d-9c40-62c20db48e36" containerID="93236ab6108a191348fb1dd806c4f24157149223223b60544bb64f324e5a9ce5" exitCode=0
Sep 19 01:16:41.374409 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:41.374060    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c" event={"ID":"9d1f64b3-110a-4a4d-9c40-62c20db48e36","Type":"ContainerDied","Data":"93236ab6108a191348fb1dd806c4f24157149223223b60544bb64f324e5a9ce5"}
Sep 19 01:16:42.381426 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:42.381394    2580 generic.go:356] "Generic (PLEG): container finished" podID="9d1f64b3-110a-4a4d-9c40-62c20db48e36" containerID="4134861f184f6a17bfc4fc2d9a5158f18b0b3a7e8b45e7278100fae89e979388" exitCode=0
Sep 19 01:16:42.381977 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:42.381448    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c" event={"ID":"9d1f64b3-110a-4a4d-9c40-62c20db48e36","Type":"ContainerDied","Data":"4134861f184f6a17bfc4fc2d9a5158f18b0b3a7e8b45e7278100fae89e979388"}
Sep 19 01:16:43.387527 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:43.387458    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-zzchf" event={"ID":"da9e4c81-6d57-4cfb-8298-433cb0339a0c","Type":"ContainerStarted","Data":"c0e53fbf9afbe4c394a5a158d4eaf51b23d3d507e15ed21f26d1dc6da26a5686"}
Sep 19 01:16:43.387969 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:43.387770    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-zzchf"
Sep 19 01:16:43.406893 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:43.406845    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-zzchf" podStartSLOduration=1.478387991 podStartE2EDuration="5.40682998s" totalImagesPullingTime="3.928441989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:16:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:16:43.406787703 +0000 UTC m=+258.313649750" watchObservedRunningTime="2026-09-19 01:16:43.40682998 +0000 UTC m=+258.313692028"
Sep 19 01:16:43.513003 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:43.512982    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c"
Sep 19 01:16:43.565550 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:43.565522    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d1f64b3-110a-4a4d-9c40-62c20db48e36-util\") pod \"9d1f64b3-110a-4a4d-9c40-62c20db48e36\" (UID: \"9d1f64b3-110a-4a4d-9c40-62c20db48e36\") "
Sep 19 01:16:43.565713 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:43.565586    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwpkz\" (UniqueName: \"kubernetes.io/projected/9d1f64b3-110a-4a4d-9c40-62c20db48e36-kube-api-access-pwpkz\") pod \"9d1f64b3-110a-4a4d-9c40-62c20db48e36\" (UID: \"9d1f64b3-110a-4a4d-9c40-62c20db48e36\") "
Sep 19 01:16:43.565713 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:43.565644    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d1f64b3-110a-4a4d-9c40-62c20db48e36-bundle\") pod \"9d1f64b3-110a-4a4d-9c40-62c20db48e36\" (UID: \"9d1f64b3-110a-4a4d-9c40-62c20db48e36\") "
Sep 19 01:16:43.566067 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:43.566033    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d1f64b3-110a-4a4d-9c40-62c20db48e36-bundle" (OuterVolumeSpecName: "bundle") pod "9d1f64b3-110a-4a4d-9c40-62c20db48e36" (UID: "9d1f64b3-110a-4a4d-9c40-62c20db48e36"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:16:43.567742 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:43.567719    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d1f64b3-110a-4a4d-9c40-62c20db48e36-kube-api-access-pwpkz" (OuterVolumeSpecName: "kube-api-access-pwpkz") pod "9d1f64b3-110a-4a4d-9c40-62c20db48e36" (UID: "9d1f64b3-110a-4a4d-9c40-62c20db48e36"). InnerVolumeSpecName "kube-api-access-pwpkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:16:43.570930 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:43.570910    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d1f64b3-110a-4a4d-9c40-62c20db48e36-util" (OuterVolumeSpecName: "util") pod "9d1f64b3-110a-4a4d-9c40-62c20db48e36" (UID: "9d1f64b3-110a-4a4d-9c40-62c20db48e36"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:16:43.666422 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:43.666355    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwpkz\" (UniqueName: \"kubernetes.io/projected/9d1f64b3-110a-4a4d-9c40-62c20db48e36-kube-api-access-pwpkz\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:16:43.666422 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:43.666382    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d1f64b3-110a-4a4d-9c40-62c20db48e36-bundle\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:16:43.666422 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:43.666392    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d1f64b3-110a-4a4d-9c40-62c20db48e36-util\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:16:44.394082 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:44.394056    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c"
Sep 19 01:16:44.394539 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:44.394079    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frww9c" event={"ID":"9d1f64b3-110a-4a4d-9c40-62c20db48e36","Type":"ContainerDied","Data":"d6919455fa24f8c23ecce92be8c3878126c5f16d02c5bca99c0e5886706211ec"}
Sep 19 01:16:44.394539 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:44.394121    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d6919455fa24f8c23ecce92be8c3878126c5f16d02c5bca99c0e5886706211ec"
Sep 19 01:16:48.248956 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:48.248921    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-f5m9m"]
Sep 19 01:16:48.249309 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:48.249227    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d1f64b3-110a-4a4d-9c40-62c20db48e36" containerName="util"
Sep 19 01:16:48.249309 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:48.249238    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d1f64b3-110a-4a4d-9c40-62c20db48e36" containerName="util"
Sep 19 01:16:48.249309 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:48.249246    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d1f64b3-110a-4a4d-9c40-62c20db48e36" containerName="extract"
Sep 19 01:16:48.249309 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:48.249251    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d1f64b3-110a-4a4d-9c40-62c20db48e36" containerName="extract"
Sep 19 01:16:48.249309 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:48.249265    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d1f64b3-110a-4a4d-9c40-62c20db48e36" containerName="pull"
Sep 19 01:16:48.249309 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:48.249270    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d1f64b3-110a-4a4d-9c40-62c20db48e36" containerName="pull"
Sep 19 01:16:48.249521 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:48.249328    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d1f64b3-110a-4a4d-9c40-62c20db48e36" containerName="extract"
Sep 19 01:16:48.254964 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:48.254946    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-f5m9m"
Sep 19 01:16:48.257755 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:48.257737    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-chr6l\""
Sep 19 01:16:48.270187 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:48.270165    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-f5m9m"]
Sep 19 01:16:48.402799 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:48.402767    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7l7m\" (UniqueName: \"kubernetes.io/projected/892cbb1a-7bfe-4743-9a45-a3e38f18ed05-kube-api-access-n7l7m\") pod \"cert-manager-55b4d954d5-f5m9m\" (UID: \"892cbb1a-7bfe-4743-9a45-a3e38f18ed05\") " pod="cert-manager/cert-manager-55b4d954d5-f5m9m"
Sep 19 01:16:48.402963 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:48.402813    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/892cbb1a-7bfe-4743-9a45-a3e38f18ed05-bound-sa-token\") pod \"cert-manager-55b4d954d5-f5m9m\" (UID: \"892cbb1a-7bfe-4743-9a45-a3e38f18ed05\") " pod="cert-manager/cert-manager-55b4d954d5-f5m9m"
Sep 19 01:16:48.503696 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:48.503617    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7l7m\" (UniqueName: \"kubernetes.io/projected/892cbb1a-7bfe-4743-9a45-a3e38f18ed05-kube-api-access-n7l7m\") pod \"cert-manager-55b4d954d5-f5m9m\" (UID: \"892cbb1a-7bfe-4743-9a45-a3e38f18ed05\") " pod="cert-manager/cert-manager-55b4d954d5-f5m9m"
Sep 19 01:16:48.503696 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:48.503674    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/892cbb1a-7bfe-4743-9a45-a3e38f18ed05-bound-sa-token\") pod \"cert-manager-55b4d954d5-f5m9m\" (UID: \"892cbb1a-7bfe-4743-9a45-a3e38f18ed05\") " pod="cert-manager/cert-manager-55b4d954d5-f5m9m"
Sep 19 01:16:48.519854 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:48.519819    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/892cbb1a-7bfe-4743-9a45-a3e38f18ed05-bound-sa-token\") pod \"cert-manager-55b4d954d5-f5m9m\" (UID: \"892cbb1a-7bfe-4743-9a45-a3e38f18ed05\") " pod="cert-manager/cert-manager-55b4d954d5-f5m9m"
Sep 19 01:16:48.519982 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:48.519900    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7l7m\" (UniqueName: \"kubernetes.io/projected/892cbb1a-7bfe-4743-9a45-a3e38f18ed05-kube-api-access-n7l7m\") pod \"cert-manager-55b4d954d5-f5m9m\" (UID: \"892cbb1a-7bfe-4743-9a45-a3e38f18ed05\") " pod="cert-manager/cert-manager-55b4d954d5-f5m9m"
Sep 19 01:16:48.563798 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:48.563775    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-f5m9m"
Sep 19 01:16:48.861170 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:48.861144    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-f5m9m"]
Sep 19 01:16:48.863863 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:16:48.863836    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod892cbb1a_7bfe_4743_9a45_a3e38f18ed05.slice/crio-2b6fef631cbe3193f5b2296bd3627cf5d19a65dd85d4754163e025aaba274dac WatchSource:0}: Error finding container 2b6fef631cbe3193f5b2296bd3627cf5d19a65dd85d4754163e025aaba274dac: Status 404 returned error can't find the container with id 2b6fef631cbe3193f5b2296bd3627cf5d19a65dd85d4754163e025aaba274dac
Sep 19 01:16:49.396983 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:49.396954    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-zzchf"
Sep 19 01:16:49.416458 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:49.416431    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-f5m9m" event={"ID":"892cbb1a-7bfe-4743-9a45-a3e38f18ed05","Type":"ContainerStarted","Data":"f3fc00c6437d16f846f2706df7ac00be541363bd15a7c6d829744713cbb845f8"}
Sep 19 01:16:49.416458 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:49.416462    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-f5m9m" event={"ID":"892cbb1a-7bfe-4743-9a45-a3e38f18ed05","Type":"ContainerStarted","Data":"2b6fef631cbe3193f5b2296bd3627cf5d19a65dd85d4754163e025aaba274dac"}
Sep 19 01:16:49.483207 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:49.483159    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-f5m9m" podStartSLOduration=1.483142908 podStartE2EDuration="1.483142908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:16:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:16:49.481135479 +0000 UTC m=+264.387997528" watchObservedRunningTime="2026-09-19 01:16:49.483142908 +0000 UTC m=+264.390004955"
Sep 19 01:16:51.030601 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:51.030571    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9kn6"]
Sep 19 01:16:51.053291 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:51.053268    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9kn6"
Sep 19 01:16:51.061313 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:51.061136    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-djpsd\""
Sep 19 01:16:51.068310 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:51.068287    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 19 01:16:51.068447 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:51.068320    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 19 01:16:51.069112 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:51.069091    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9kn6"]
Sep 19 01:16:51.126096 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:51.126065    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3d3d81d4-e171-4c3b-a1e5-135280d40915-tmp\") pod \"openshift-lws-operator-fd8ccff4c-j9kn6\" (UID: \"3d3d81d4-e171-4c3b-a1e5-135280d40915\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9kn6"
Sep 19 01:16:51.126220 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:51.126124    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xfkt\" (UniqueName: \"kubernetes.io/projected/3d3d81d4-e171-4c3b-a1e5-135280d40915-kube-api-access-8xfkt\") pod \"openshift-lws-operator-fd8ccff4c-j9kn6\" (UID: \"3d3d81d4-e171-4c3b-a1e5-135280d40915\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9kn6"
Sep 19 01:16:51.227332 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:51.227303    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xfkt\" (UniqueName: \"kubernetes.io/projected/3d3d81d4-e171-4c3b-a1e5-135280d40915-kube-api-access-8xfkt\") pod \"openshift-lws-operator-fd8ccff4c-j9kn6\" (UID: \"3d3d81d4-e171-4c3b-a1e5-135280d40915\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9kn6"
Sep 19 01:16:51.227483 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:51.227390    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3d3d81d4-e171-4c3b-a1e5-135280d40915-tmp\") pod \"openshift-lws-operator-fd8ccff4c-j9kn6\" (UID: \"3d3d81d4-e171-4c3b-a1e5-135280d40915\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9kn6"
Sep 19 01:16:51.227773 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:51.227752    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3d3d81d4-e171-4c3b-a1e5-135280d40915-tmp\") pod \"openshift-lws-operator-fd8ccff4c-j9kn6\" (UID: \"3d3d81d4-e171-4c3b-a1e5-135280d40915\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9kn6"
Sep 19 01:16:51.239168 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:51.239143    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xfkt\" (UniqueName: \"kubernetes.io/projected/3d3d81d4-e171-4c3b-a1e5-135280d40915-kube-api-access-8xfkt\") pod \"openshift-lws-operator-fd8ccff4c-j9kn6\" (UID: \"3d3d81d4-e171-4c3b-a1e5-135280d40915\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9kn6"
Sep 19 01:16:51.367895 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:51.367831    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9kn6"
Sep 19 01:16:51.665337 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:51.665226    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9kn6"]
Sep 19 01:16:51.667921 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:16:51.667879    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d3d81d4_e171_4c3b_a1e5_135280d40915.slice/crio-906b260b8c10c3e906e0087c347a26f292725f625bcb1cb11ea8b1393983f40f WatchSource:0}: Error finding container 906b260b8c10c3e906e0087c347a26f292725f625bcb1cb11ea8b1393983f40f: Status 404 returned error can't find the container with id 906b260b8c10c3e906e0087c347a26f292725f625bcb1cb11ea8b1393983f40f
Sep 19 01:16:52.433054 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:52.433018    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9kn6" event={"ID":"3d3d81d4-e171-4c3b-a1e5-135280d40915","Type":"ContainerStarted","Data":"906b260b8c10c3e906e0087c347a26f292725f625bcb1cb11ea8b1393983f40f"}
Sep 19 01:16:54.446142 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:54.446110    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9kn6" event={"ID":"3d3d81d4-e171-4c3b-a1e5-135280d40915","Type":"ContainerStarted","Data":"c22a8272a445bdb9c9384cbeb82502961b3aa053f19d8232bf784d6b9ec6ddcd"}
Sep 19 01:16:54.465677 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:54.465632    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j9kn6" podStartSLOduration=0.815120952 podStartE2EDuration="3.465615209s" totalImagesPullingTime="2.650494257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:16:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:16:54.463664587 +0000 UTC m=+269.370526635" watchObservedRunningTime="2026-09-19 01:16:54.465615209 +0000 UTC m=+269.372477257"
Sep 19 01:16:56.875005 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:56.874965    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9"]
Sep 19 01:16:56.880414 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:56.880392    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9"
Sep 19 01:16:56.883179 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:56.883154    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 19 01:16:56.883755 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:56.883739    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 19 01:16:56.883827 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:56.883740    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kfj97\""
Sep 19 01:16:56.888172 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:56.887912    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9"]
Sep 19 01:16:56.979718 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:56.979682    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/398c6913-9926-4e4b-b5ea-1cf2132d8da8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9\" (UID: \"398c6913-9926-4e4b-b5ea-1cf2132d8da8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9"
Sep 19 01:16:56.979874 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:56.979787    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vgdj\" (UniqueName: \"kubernetes.io/projected/398c6913-9926-4e4b-b5ea-1cf2132d8da8-kube-api-access-6vgdj\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9\" (UID: \"398c6913-9926-4e4b-b5ea-1cf2132d8da8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9"
Sep 19 01:16:56.979874 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:56.979831    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/398c6913-9926-4e4b-b5ea-1cf2132d8da8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9\" (UID: \"398c6913-9926-4e4b-b5ea-1cf2132d8da8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9"
Sep 19 01:16:57.081174 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:57.081140    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vgdj\" (UniqueName: \"kubernetes.io/projected/398c6913-9926-4e4b-b5ea-1cf2132d8da8-kube-api-access-6vgdj\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9\" (UID: \"398c6913-9926-4e4b-b5ea-1cf2132d8da8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9"
Sep 19 01:16:57.081359 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:57.081188    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/398c6913-9926-4e4b-b5ea-1cf2132d8da8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9\" (UID: \"398c6913-9926-4e4b-b5ea-1cf2132d8da8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9"
Sep 19 01:16:57.081359 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:57.081216    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/398c6913-9926-4e4b-b5ea-1cf2132d8da8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9\" (UID: \"398c6913-9926-4e4b-b5ea-1cf2132d8da8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9"
Sep 19 01:16:57.081596 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:57.081577    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/398c6913-9926-4e4b-b5ea-1cf2132d8da8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9\" (UID: \"398c6913-9926-4e4b-b5ea-1cf2132d8da8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9"
Sep 19 01:16:57.081679 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:57.081629    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/398c6913-9926-4e4b-b5ea-1cf2132d8da8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9\" (UID: \"398c6913-9926-4e4b-b5ea-1cf2132d8da8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9"
Sep 19 01:16:57.089217 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:57.089189    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vgdj\" (UniqueName: \"kubernetes.io/projected/398c6913-9926-4e4b-b5ea-1cf2132d8da8-kube-api-access-6vgdj\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9\" (UID: \"398c6913-9926-4e4b-b5ea-1cf2132d8da8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9"
Sep 19 01:16:57.191258 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:57.191168    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9"
Sep 19 01:16:57.461688 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:57.461659    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9"]
Sep 19 01:16:57.464310 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:16:57.464282    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod398c6913_9926_4e4b_b5ea_1cf2132d8da8.slice/crio-ac860dcb2519d0fa2b281896ac0b3a1f64f1a2bd1190225cd1a2c43f55daff52 WatchSource:0}: Error finding container ac860dcb2519d0fa2b281896ac0b3a1f64f1a2bd1190225cd1a2c43f55daff52: Status 404 returned error can't find the container with id ac860dcb2519d0fa2b281896ac0b3a1f64f1a2bd1190225cd1a2c43f55daff52
Sep 19 01:16:58.466038 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:58.465952    2580 generic.go:356] "Generic (PLEG): container finished" podID="398c6913-9926-4e4b-b5ea-1cf2132d8da8" containerID="c4906c668c06617f05a5676657b07baa93bbd7dd00f040c93c2521c8e160e5f6" exitCode=0
Sep 19 01:16:58.466414 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:58.466042    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9" event={"ID":"398c6913-9926-4e4b-b5ea-1cf2132d8da8","Type":"ContainerDied","Data":"c4906c668c06617f05a5676657b07baa93bbd7dd00f040c93c2521c8e160e5f6"}
Sep 19 01:16:58.466414 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:58.466102    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9" event={"ID":"398c6913-9926-4e4b-b5ea-1cf2132d8da8","Type":"ContainerStarted","Data":"ac860dcb2519d0fa2b281896ac0b3a1f64f1a2bd1190225cd1a2c43f55daff52"}
Sep 19 01:16:59.473043 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:59.472951    2580 generic.go:356] "Generic (PLEG): container finished" podID="398c6913-9926-4e4b-b5ea-1cf2132d8da8" containerID="9f9699749ee4e05f2ab4932d5dd4f8b019e73351eb795035ce899647dccc8274" exitCode=0
Sep 19 01:16:59.473043 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:16:59.473032    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9" event={"ID":"398c6913-9926-4e4b-b5ea-1cf2132d8da8","Type":"ContainerDied","Data":"9f9699749ee4e05f2ab4932d5dd4f8b019e73351eb795035ce899647dccc8274"}
Sep 19 01:17:00.480142 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:00.480109    2580 generic.go:356] "Generic (PLEG): container finished" podID="398c6913-9926-4e4b-b5ea-1cf2132d8da8" containerID="efe6acf9cc81acd08cb6ace0e925905e28ae4acd90a5b76cb031db030f18ddaf" exitCode=0
Sep 19 01:17:00.480142 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:00.480147    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9" event={"ID":"398c6913-9926-4e4b-b5ea-1cf2132d8da8","Type":"ContainerDied","Data":"efe6acf9cc81acd08cb6ace0e925905e28ae4acd90a5b76cb031db030f18ddaf"}
Sep 19 01:17:01.613617 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:01.613588    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9"
Sep 19 01:17:01.718343 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:01.718317    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/398c6913-9926-4e4b-b5ea-1cf2132d8da8-bundle\") pod \"398c6913-9926-4e4b-b5ea-1cf2132d8da8\" (UID: \"398c6913-9926-4e4b-b5ea-1cf2132d8da8\") "
Sep 19 01:17:01.718553 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:01.718355    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/398c6913-9926-4e4b-b5ea-1cf2132d8da8-util\") pod \"398c6913-9926-4e4b-b5ea-1cf2132d8da8\" (UID: \"398c6913-9926-4e4b-b5ea-1cf2132d8da8\") "
Sep 19 01:17:01.718553 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:01.718403    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vgdj\" (UniqueName: \"kubernetes.io/projected/398c6913-9926-4e4b-b5ea-1cf2132d8da8-kube-api-access-6vgdj\") pod \"398c6913-9926-4e4b-b5ea-1cf2132d8da8\" (UID: \"398c6913-9926-4e4b-b5ea-1cf2132d8da8\") "
Sep 19 01:17:01.719028 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:01.718990    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/398c6913-9926-4e4b-b5ea-1cf2132d8da8-bundle" (OuterVolumeSpecName: "bundle") pod "398c6913-9926-4e4b-b5ea-1cf2132d8da8" (UID: "398c6913-9926-4e4b-b5ea-1cf2132d8da8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:17:01.720510 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:01.720474    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/398c6913-9926-4e4b-b5ea-1cf2132d8da8-kube-api-access-6vgdj" (OuterVolumeSpecName: "kube-api-access-6vgdj") pod "398c6913-9926-4e4b-b5ea-1cf2132d8da8" (UID: "398c6913-9926-4e4b-b5ea-1cf2132d8da8"). InnerVolumeSpecName "kube-api-access-6vgdj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:17:01.724063 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:01.724025    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/398c6913-9926-4e4b-b5ea-1cf2132d8da8-util" (OuterVolumeSpecName: "util") pod "398c6913-9926-4e4b-b5ea-1cf2132d8da8" (UID: "398c6913-9926-4e4b-b5ea-1cf2132d8da8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:17:01.819891 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:01.819832    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/398c6913-9926-4e4b-b5ea-1cf2132d8da8-bundle\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:17:01.819891 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:01.819852    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/398c6913-9926-4e4b-b5ea-1cf2132d8da8-util\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:17:01.819891 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:01.819864    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vgdj\" (UniqueName: \"kubernetes.io/projected/398c6913-9926-4e4b-b5ea-1cf2132d8da8-kube-api-access-6vgdj\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:17:02.490910 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:02.490881    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9"
Sep 19 01:17:02.490910 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:02.490894    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrwmc9" event={"ID":"398c6913-9926-4e4b-b5ea-1cf2132d8da8","Type":"ContainerDied","Data":"ac860dcb2519d0fa2b281896ac0b3a1f64f1a2bd1190225cd1a2c43f55daff52"}
Sep 19 01:17:02.491114 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:02.490934    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ac860dcb2519d0fa2b281896ac0b3a1f64f1a2bd1190225cd1a2c43f55daff52"
Sep 19 01:17:14.268906 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.268870    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8"]
Sep 19 01:17:14.269300 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.269221    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="398c6913-9926-4e4b-b5ea-1cf2132d8da8" containerName="extract"
Sep 19 01:17:14.269300 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.269233    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="398c6913-9926-4e4b-b5ea-1cf2132d8da8" containerName="extract"
Sep 19 01:17:14.269300 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.269248    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="398c6913-9926-4e4b-b5ea-1cf2132d8da8" containerName="pull"
Sep 19 01:17:14.269300 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.269255    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="398c6913-9926-4e4b-b5ea-1cf2132d8da8" containerName="pull"
Sep 19 01:17:14.269300 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.269269    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="398c6913-9926-4e4b-b5ea-1cf2132d8da8" containerName="util"
Sep 19 01:17:14.269300 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.269275    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="398c6913-9926-4e4b-b5ea-1cf2132d8da8" containerName="util"
Sep 19 01:17:14.269488 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.269322    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="398c6913-9926-4e4b-b5ea-1cf2132d8da8" containerName="extract"
Sep 19 01:17:14.272507 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.272482    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8"
Sep 19 01:17:14.276878 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.276859    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 19 01:17:14.277399 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.277381    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kfj97\""
Sep 19 01:17:14.279453 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.279433    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 19 01:17:14.290156 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.290132    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8"]
Sep 19 01:17:14.423866 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.423828    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7efc7ff0-09c7-4872-9ae3-bce6051e9fb8-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8\" (UID: \"7efc7ff0-09c7-4872-9ae3-bce6051e9fb8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8"
Sep 19 01:17:14.423866 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.423872    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7efc7ff0-09c7-4872-9ae3-bce6051e9fb8-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8\" (UID: \"7efc7ff0-09c7-4872-9ae3-bce6051e9fb8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8"
Sep 19 01:17:14.424104 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.423898    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54rw5\" (UniqueName: \"kubernetes.io/projected/7efc7ff0-09c7-4872-9ae3-bce6051e9fb8-kube-api-access-54rw5\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8\" (UID: \"7efc7ff0-09c7-4872-9ae3-bce6051e9fb8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8"
Sep 19 01:17:14.524897 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.524825    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54rw5\" (UniqueName: \"kubernetes.io/projected/7efc7ff0-09c7-4872-9ae3-bce6051e9fb8-kube-api-access-54rw5\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8\" (UID: \"7efc7ff0-09c7-4872-9ae3-bce6051e9fb8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8"
Sep 19 01:17:14.525033 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.524910    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7efc7ff0-09c7-4872-9ae3-bce6051e9fb8-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8\" (UID: \"7efc7ff0-09c7-4872-9ae3-bce6051e9fb8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8"
Sep 19 01:17:14.525033 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.524930    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7efc7ff0-09c7-4872-9ae3-bce6051e9fb8-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8\" (UID: \"7efc7ff0-09c7-4872-9ae3-bce6051e9fb8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8"
Sep 19 01:17:14.525230 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.525215    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7efc7ff0-09c7-4872-9ae3-bce6051e9fb8-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8\" (UID: \"7efc7ff0-09c7-4872-9ae3-bce6051e9fb8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8"
Sep 19 01:17:14.525302 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.525285    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7efc7ff0-09c7-4872-9ae3-bce6051e9fb8-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8\" (UID: \"7efc7ff0-09c7-4872-9ae3-bce6051e9fb8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8"
Sep 19 01:17:14.534920 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.534894    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54rw5\" (UniqueName: \"kubernetes.io/projected/7efc7ff0-09c7-4872-9ae3-bce6051e9fb8-kube-api-access-54rw5\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8\" (UID: \"7efc7ff0-09c7-4872-9ae3-bce6051e9fb8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8"
Sep 19 01:17:14.582167 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.582132    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8"
Sep 19 01:17:14.770099 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.770065    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z"]
Sep 19 01:17:14.773693 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.773671    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z"
Sep 19 01:17:14.781271 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.781250    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 19 01:17:14.799743 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.799724    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 19 01:17:14.804313 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.804288    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z"]
Sep 19 01:17:14.807208 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.807193    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 19 01:17:14.807330 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.807302    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 19 01:17:14.808050 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.808031    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-h4tpm\""
Sep 19 01:17:14.827456 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.827432    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2015c019-405a-4565-9c96-8ddda261051c-webhook-cert\") pod \"opendatahub-operator-controller-manager-748475db69-5dq5z\" (UID: \"2015c019-405a-4565-9c96-8ddda261051c\") " pod="opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z"
Sep 19 01:17:14.827589 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.827485    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2015c019-405a-4565-9c96-8ddda261051c-tmp\") pod \"opendatahub-operator-controller-manager-748475db69-5dq5z\" (UID: \"2015c019-405a-4565-9c96-8ddda261051c\") " pod="opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z"
Sep 19 01:17:14.827589 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.827521    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdrhr\" (UniqueName: \"kubernetes.io/projected/2015c019-405a-4565-9c96-8ddda261051c-kube-api-access-zdrhr\") pod \"opendatahub-operator-controller-manager-748475db69-5dq5z\" (UID: \"2015c019-405a-4565-9c96-8ddda261051c\") " pod="opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z"
Sep 19 01:17:14.827589 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.827581    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2015c019-405a-4565-9c96-8ddda261051c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-748475db69-5dq5z\" (UID: \"2015c019-405a-4565-9c96-8ddda261051c\") " pod="opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z"
Sep 19 01:17:14.827694 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.827606    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2015c019-405a-4565-9c96-8ddda261051c-manifests\") pod \"opendatahub-operator-controller-manager-748475db69-5dq5z\" (UID: \"2015c019-405a-4565-9c96-8ddda261051c\") " pod="opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z"
Sep 19 01:17:14.893782 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.893759    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8"]
Sep 19 01:17:14.896623 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:17:14.896599    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7efc7ff0_09c7_4872_9ae3_bce6051e9fb8.slice/crio-f01fbef253aa1ac9168a78b430396f1d3a10a25cd5a660226565f19a7704e328 WatchSource:0}: Error finding container f01fbef253aa1ac9168a78b430396f1d3a10a25cd5a660226565f19a7704e328: Status 404 returned error can't find the container with id f01fbef253aa1ac9168a78b430396f1d3a10a25cd5a660226565f19a7704e328
Sep 19 01:17:14.928660 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.928638    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2015c019-405a-4565-9c96-8ddda261051c-webhook-cert\") pod \"opendatahub-operator-controller-manager-748475db69-5dq5z\" (UID: \"2015c019-405a-4565-9c96-8ddda261051c\") " pod="opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z"
Sep 19 01:17:14.928757 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.928692    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2015c019-405a-4565-9c96-8ddda261051c-tmp\") pod \"opendatahub-operator-controller-manager-748475db69-5dq5z\" (UID: \"2015c019-405a-4565-9c96-8ddda261051c\") " pod="opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z"
Sep 19 01:17:14.928901 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.928862    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdrhr\" (UniqueName: \"kubernetes.io/projected/2015c019-405a-4565-9c96-8ddda261051c-kube-api-access-zdrhr\") pod \"opendatahub-operator-controller-manager-748475db69-5dq5z\" (UID: \"2015c019-405a-4565-9c96-8ddda261051c\") " pod="opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z"
Sep 19 01:17:14.928986 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.928962    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2015c019-405a-4565-9c96-8ddda261051c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-748475db69-5dq5z\" (UID: \"2015c019-405a-4565-9c96-8ddda261051c\") " pod="opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z"
Sep 19 01:17:14.929107 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.929005    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2015c019-405a-4565-9c96-8ddda261051c-manifests\") pod \"opendatahub-operator-controller-manager-748475db69-5dq5z\" (UID: \"2015c019-405a-4565-9c96-8ddda261051c\") " pod="opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z"
Sep 19 01:17:14.929329 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.929311    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2015c019-405a-4565-9c96-8ddda261051c-manifests\") pod \"opendatahub-operator-controller-manager-748475db69-5dq5z\" (UID: \"2015c019-405a-4565-9c96-8ddda261051c\") " pod="opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z"
Sep 19 01:17:14.930714 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.930697    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2015c019-405a-4565-9c96-8ddda261051c-tmp\") pod \"opendatahub-operator-controller-manager-748475db69-5dq5z\" (UID: \"2015c019-405a-4565-9c96-8ddda261051c\") " pod="opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z"
Sep 19 01:17:14.931018 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.931003    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2015c019-405a-4565-9c96-8ddda261051c-webhook-cert\") pod \"opendatahub-operator-controller-manager-748475db69-5dq5z\" (UID: \"2015c019-405a-4565-9c96-8ddda261051c\") " pod="opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z"
Sep 19 01:17:14.931107 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.931089    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2015c019-405a-4565-9c96-8ddda261051c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-748475db69-5dq5z\" (UID: \"2015c019-405a-4565-9c96-8ddda261051c\") " pod="opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z"
Sep 19 01:17:14.938569 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:14.938543    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdrhr\" (UniqueName: \"kubernetes.io/projected/2015c019-405a-4565-9c96-8ddda261051c-kube-api-access-zdrhr\") pod \"opendatahub-operator-controller-manager-748475db69-5dq5z\" (UID: \"2015c019-405a-4565-9c96-8ddda261051c\") " pod="opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z"
Sep 19 01:17:15.084974 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:15.084900    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z"
Sep 19 01:17:15.371229 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:15.371206    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z"]
Sep 19 01:17:15.372904 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:17:15.372875    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2015c019_405a_4565_9c96_8ddda261051c.slice/crio-1d47d6b475b97dc17035332d0f3f069438afca3f41774b4785368dac60c26ab9 WatchSource:0}: Error finding container 1d47d6b475b97dc17035332d0f3f069438afca3f41774b4785368dac60c26ab9: Status 404 returned error can't find the container with id 1d47d6b475b97dc17035332d0f3f069438afca3f41774b4785368dac60c26ab9
Sep 19 01:17:15.560273 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:15.560239    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z" event={"ID":"2015c019-405a-4565-9c96-8ddda261051c","Type":"ContainerStarted","Data":"1d47d6b475b97dc17035332d0f3f069438afca3f41774b4785368dac60c26ab9"}
Sep 19 01:17:15.561666 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:15.561637    2580 generic.go:356] "Generic (PLEG): container finished" podID="7efc7ff0-09c7-4872-9ae3-bce6051e9fb8" containerID="0c07bac3b2a125dcdf754ca3e022866e9fcd3aacbb79e1397c473af2e0605def" exitCode=0
Sep 19 01:17:15.561779 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:15.561671    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8" event={"ID":"7efc7ff0-09c7-4872-9ae3-bce6051e9fb8","Type":"ContainerDied","Data":"0c07bac3b2a125dcdf754ca3e022866e9fcd3aacbb79e1397c473af2e0605def"}
Sep 19 01:17:15.561779 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:15.561712    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8" event={"ID":"7efc7ff0-09c7-4872-9ae3-bce6051e9fb8","Type":"ContainerStarted","Data":"f01fbef253aa1ac9168a78b430396f1d3a10a25cd5a660226565f19a7704e328"}
Sep 19 01:17:17.575987 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:17.575945    2580 generic.go:356] "Generic (PLEG): container finished" podID="7efc7ff0-09c7-4872-9ae3-bce6051e9fb8" containerID="f96e293b614285ef4c91ec860c321a54d7c01e0c6462c4162b888c178631283f" exitCode=0
Sep 19 01:17:17.576554 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:17.576028    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8" event={"ID":"7efc7ff0-09c7-4872-9ae3-bce6051e9fb8","Type":"ContainerDied","Data":"f96e293b614285ef4c91ec860c321a54d7c01e0c6462c4162b888c178631283f"}
Sep 19 01:17:19.587423 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:19.587384    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z" event={"ID":"2015c019-405a-4565-9c96-8ddda261051c","Type":"ContainerStarted","Data":"b71783232289ca5f8dca77dda4f3096c018909ac720bb5242e0277de3aeb598f"}
Sep 19 01:17:19.589862 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:19.589839    2580 generic.go:356] "Generic (PLEG): container finished" podID="7efc7ff0-09c7-4872-9ae3-bce6051e9fb8" containerID="ea29272eb9dbabd165ea96164913480e9479f205abb468ac27275d88c7f9ae3e" exitCode=0
Sep 19 01:17:19.589980 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:19.589912    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8" event={"ID":"7efc7ff0-09c7-4872-9ae3-bce6051e9fb8","Type":"ContainerDied","Data":"ea29272eb9dbabd165ea96164913480e9479f205abb468ac27275d88c7f9ae3e"}
Sep 19 01:17:20.595783 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:20.595750    2580 generic.go:356] "Generic (PLEG): container finished" podID="2015c019-405a-4565-9c96-8ddda261051c" containerID="b71783232289ca5f8dca77dda4f3096c018909ac720bb5242e0277de3aeb598f" exitCode=0
Sep 19 01:17:20.596230 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:20.595844    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z" event={"ID":"2015c019-405a-4565-9c96-8ddda261051c","Type":"ContainerDied","Data":"b71783232289ca5f8dca77dda4f3096c018909ac720bb5242e0277de3aeb598f"}
Sep 19 01:17:20.732685 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:20.732655    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8"
Sep 19 01:17:20.779326 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:20.779302    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54rw5\" (UniqueName: \"kubernetes.io/projected/7efc7ff0-09c7-4872-9ae3-bce6051e9fb8-kube-api-access-54rw5\") pod \"7efc7ff0-09c7-4872-9ae3-bce6051e9fb8\" (UID: \"7efc7ff0-09c7-4872-9ae3-bce6051e9fb8\") "
Sep 19 01:17:20.779470 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:20.779345    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7efc7ff0-09c7-4872-9ae3-bce6051e9fb8-util\") pod \"7efc7ff0-09c7-4872-9ae3-bce6051e9fb8\" (UID: \"7efc7ff0-09c7-4872-9ae3-bce6051e9fb8\") "
Sep 19 01:17:20.779470 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:20.779388    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7efc7ff0-09c7-4872-9ae3-bce6051e9fb8-bundle\") pod \"7efc7ff0-09c7-4872-9ae3-bce6051e9fb8\" (UID: \"7efc7ff0-09c7-4872-9ae3-bce6051e9fb8\") "
Sep 19 01:17:20.780128 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:20.780074    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7efc7ff0-09c7-4872-9ae3-bce6051e9fb8-bundle" (OuterVolumeSpecName: "bundle") pod "7efc7ff0-09c7-4872-9ae3-bce6051e9fb8" (UID: "7efc7ff0-09c7-4872-9ae3-bce6051e9fb8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:17:20.781614 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:20.781580    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7efc7ff0-09c7-4872-9ae3-bce6051e9fb8-kube-api-access-54rw5" (OuterVolumeSpecName: "kube-api-access-54rw5") pod "7efc7ff0-09c7-4872-9ae3-bce6051e9fb8" (UID: "7efc7ff0-09c7-4872-9ae3-bce6051e9fb8"). InnerVolumeSpecName "kube-api-access-54rw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:17:20.783988 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:20.783948    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7efc7ff0-09c7-4872-9ae3-bce6051e9fb8-util" (OuterVolumeSpecName: "util") pod "7efc7ff0-09c7-4872-9ae3-bce6051e9fb8" (UID: "7efc7ff0-09c7-4872-9ae3-bce6051e9fb8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:17:20.880133 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:20.880050    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7efc7ff0-09c7-4872-9ae3-bce6051e9fb8-bundle\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:17:20.880133 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:20.880094    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54rw5\" (UniqueName: \"kubernetes.io/projected/7efc7ff0-09c7-4872-9ae3-bce6051e9fb8-kube-api-access-54rw5\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:17:20.880133 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:20.880104    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7efc7ff0-09c7-4872-9ae3-bce6051e9fb8-util\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:17:21.602351 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:21.602323    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8"
Sep 19 01:17:21.602838 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:21.602320    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0zqds8" event={"ID":"7efc7ff0-09c7-4872-9ae3-bce6051e9fb8","Type":"ContainerDied","Data":"f01fbef253aa1ac9168a78b430396f1d3a10a25cd5a660226565f19a7704e328"}
Sep 19 01:17:21.602838 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:21.602447    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f01fbef253aa1ac9168a78b430396f1d3a10a25cd5a660226565f19a7704e328"
Sep 19 01:17:21.604677 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:21.604651    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z" event={"ID":"2015c019-405a-4565-9c96-8ddda261051c","Type":"ContainerStarted","Data":"e85474d2f85e64b1ea28dee425b836cf29b244ef86dacd8c9e37ee4151f40f27"}
Sep 19 01:17:21.604991 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:21.604971    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z"
Sep 19 01:17:21.633108 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:21.633063    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z" podStartSLOduration=2.941325216 podStartE2EDuration="7.63305115s" totalImagesPullingTime="3.654312125s" totalInitContainerRuntime="1.037413809s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:17:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:17:21.631043032 +0000 UTC m=+296.537905081" watchObservedRunningTime="2026-09-19 01:17:21.63305115 +0000 UTC m=+296.539913197"
Sep 19 01:17:27.007035 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:27.007003    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f6bdh_74c8e2f6-1b01-4bd1-b491-ad407a15a109/console-operator/1.log"
Sep 19 01:17:27.013477 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:27.013455    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f6bdh_74c8e2f6-1b01-4bd1-b491-ad407a15a109/console-operator/1.log"
Sep 19 01:17:27.020308 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:27.020286    2580 kubelet.go:1648] "Image garbage collection succeeded"
Sep 19 01:17:32.611907 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:32.611879    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-748475db69-5dq5z"
Sep 19 01:17:35.009148 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.009113    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5c6796f956-wzrsb"]
Sep 19 01:17:35.009540 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.009434    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7efc7ff0-09c7-4872-9ae3-bce6051e9fb8" containerName="pull"
Sep 19 01:17:35.009540 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.009444    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7efc7ff0-09c7-4872-9ae3-bce6051e9fb8" containerName="pull"
Sep 19 01:17:35.009540 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.009460    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7efc7ff0-09c7-4872-9ae3-bce6051e9fb8" containerName="extract"
Sep 19 01:17:35.009540 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.009465    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7efc7ff0-09c7-4872-9ae3-bce6051e9fb8" containerName="extract"
Sep 19 01:17:35.009540 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.009475    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7efc7ff0-09c7-4872-9ae3-bce6051e9fb8" containerName="util"
Sep 19 01:17:35.009540 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.009482    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7efc7ff0-09c7-4872-9ae3-bce6051e9fb8" containerName="util"
Sep 19 01:17:35.009746 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.009560    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7efc7ff0-09c7-4872-9ae3-bce6051e9fb8" containerName="extract"
Sep 19 01:17:35.012634 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.012610    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c6796f956-wzrsb"
Sep 19 01:17:35.017979 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.017955    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 19 01:17:35.025138 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.025119    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 19 01:17:35.025969 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.025947    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c6796f956-wzrsb"]
Sep 19 01:17:35.031841 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.031823    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-6k42h\""
Sep 19 01:17:35.102274 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.102247    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db018a41-4085-448a-bcae-81fe88b9f53a-tmp\") pod \"kube-auth-proxy-5c6796f956-wzrsb\" (UID: \"db018a41-4085-448a-bcae-81fe88b9f53a\") " pod="openshift-ingress/kube-auth-proxy-5c6796f956-wzrsb"
Sep 19 01:17:35.102406 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.102289    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db018a41-4085-448a-bcae-81fe88b9f53a-tls-certs\") pod \"kube-auth-proxy-5c6796f956-wzrsb\" (UID: \"db018a41-4085-448a-bcae-81fe88b9f53a\") " pod="openshift-ingress/kube-auth-proxy-5c6796f956-wzrsb"
Sep 19 01:17:35.102406 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.102354    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zkvx\" (UniqueName: \"kubernetes.io/projected/db018a41-4085-448a-bcae-81fe88b9f53a-kube-api-access-5zkvx\") pod \"kube-auth-proxy-5c6796f956-wzrsb\" (UID: \"db018a41-4085-448a-bcae-81fe88b9f53a\") " pod="openshift-ingress/kube-auth-proxy-5c6796f956-wzrsb"
Sep 19 01:17:35.203242 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.203212    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db018a41-4085-448a-bcae-81fe88b9f53a-tls-certs\") pod \"kube-auth-proxy-5c6796f956-wzrsb\" (UID: \"db018a41-4085-448a-bcae-81fe88b9f53a\") " pod="openshift-ingress/kube-auth-proxy-5c6796f956-wzrsb"
Sep 19 01:17:35.203365 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.203271    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zkvx\" (UniqueName: \"kubernetes.io/projected/db018a41-4085-448a-bcae-81fe88b9f53a-kube-api-access-5zkvx\") pod \"kube-auth-proxy-5c6796f956-wzrsb\" (UID: \"db018a41-4085-448a-bcae-81fe88b9f53a\") " pod="openshift-ingress/kube-auth-proxy-5c6796f956-wzrsb"
Sep 19 01:17:35.203365 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.203343    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db018a41-4085-448a-bcae-81fe88b9f53a-tmp\") pod \"kube-auth-proxy-5c6796f956-wzrsb\" (UID: \"db018a41-4085-448a-bcae-81fe88b9f53a\") " pod="openshift-ingress/kube-auth-proxy-5c6796f956-wzrsb"
Sep 19 01:17:35.205483 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.205464    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db018a41-4085-448a-bcae-81fe88b9f53a-tmp\") pod \"kube-auth-proxy-5c6796f956-wzrsb\" (UID: \"db018a41-4085-448a-bcae-81fe88b9f53a\") " pod="openshift-ingress/kube-auth-proxy-5c6796f956-wzrsb"
Sep 19 01:17:35.205764 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.205743    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db018a41-4085-448a-bcae-81fe88b9f53a-tls-certs\") pod \"kube-auth-proxy-5c6796f956-wzrsb\" (UID: \"db018a41-4085-448a-bcae-81fe88b9f53a\") " pod="openshift-ingress/kube-auth-proxy-5c6796f956-wzrsb"
Sep 19 01:17:35.257672 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.257651    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zkvx\" (UniqueName: \"kubernetes.io/projected/db018a41-4085-448a-bcae-81fe88b9f53a-kube-api-access-5zkvx\") pod \"kube-auth-proxy-5c6796f956-wzrsb\" (UID: \"db018a41-4085-448a-bcae-81fe88b9f53a\") " pod="openshift-ingress/kube-auth-proxy-5c6796f956-wzrsb"
Sep 19 01:17:35.321704 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.321629    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c6796f956-wzrsb"
Sep 19 01:17:35.603775 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.603750    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c6796f956-wzrsb"]
Sep 19 01:17:35.606302 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:17:35.606273    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb018a41_4085_448a_bcae_81fe88b9f53a.slice/crio-b380884c135c20b7eb0048efc29f0e3f0b495711198e5479906806d09e9545bd WatchSource:0}: Error finding container b380884c135c20b7eb0048efc29f0e3f0b495711198e5479906806d09e9545bd: Status 404 returned error can't find the container with id b380884c135c20b7eb0048efc29f0e3f0b495711198e5479906806d09e9545bd
Sep 19 01:17:35.608268 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.608252    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 01:17:35.680089 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:35.680055    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c6796f956-wzrsb" event={"ID":"db018a41-4085-448a-bcae-81fe88b9f53a","Type":"ContainerStarted","Data":"b380884c135c20b7eb0048efc29f0e3f0b495711198e5479906806d09e9545bd"}
Sep 19 01:17:36.775838 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:36.775803    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"]
Sep 19 01:17:36.779880 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:36.779856    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:36.787092 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:36.786882    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 19 01:17:36.787207 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:36.787111    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 19 01:17:36.787207 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:36.787161    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 19 01:17:36.787484 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:36.787466    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jbgkf\""
Sep 19 01:17:36.794399 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:36.794315    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 19 01:17:36.821160 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:36.821097    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"]
Sep 19 01:17:36.919824 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:36.919794    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/280fc070-bbb9-44f4-92b4-a4cd192e00c5-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-pz99g\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:36.920007 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:36.919869    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/280fc070-bbb9-44f4-92b4-a4cd192e00c5-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-pz99g\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:36.920007 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:36.919898    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/280fc070-bbb9-44f4-92b4-a4cd192e00c5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-pz99g\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:36.920007 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:36.919921    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/280fc070-bbb9-44f4-92b4-a4cd192e00c5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-pz99g\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:36.920007 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:36.919964    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/280fc070-bbb9-44f4-92b4-a4cd192e00c5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-pz99g\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:36.920230 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:36.920026    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/280fc070-bbb9-44f4-92b4-a4cd192e00c5-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-pz99g\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:36.920230 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:36.920061    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s58d\" (UniqueName: \"kubernetes.io/projected/280fc070-bbb9-44f4-92b4-a4cd192e00c5-kube-api-access-2s58d\") pod \"istiod-openshift-gateway-7b7995d5cd-pz99g\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:37.020547 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:37.020493    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/280fc070-bbb9-44f4-92b4-a4cd192e00c5-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-pz99g\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:37.020724 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:37.020556    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/280fc070-bbb9-44f4-92b4-a4cd192e00c5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-pz99g\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:37.020724 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:37.020584    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/280fc070-bbb9-44f4-92b4-a4cd192e00c5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-pz99g\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:37.020724 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:37.020620    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/280fc070-bbb9-44f4-92b4-a4cd192e00c5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-pz99g\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:37.020724 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:37.020677    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/280fc070-bbb9-44f4-92b4-a4cd192e00c5-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-pz99g\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:37.020724 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:37.020708    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s58d\" (UniqueName: \"kubernetes.io/projected/280fc070-bbb9-44f4-92b4-a4cd192e00c5-kube-api-access-2s58d\") pod \"istiod-openshift-gateway-7b7995d5cd-pz99g\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:37.021004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:37.020756    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/280fc070-bbb9-44f4-92b4-a4cd192e00c5-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-pz99g\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:37.021621 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:37.021592    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/280fc070-bbb9-44f4-92b4-a4cd192e00c5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-pz99g\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:37.023604 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:37.023583    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/280fc070-bbb9-44f4-92b4-a4cd192e00c5-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-pz99g\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:37.023692 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:37.023602    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/280fc070-bbb9-44f4-92b4-a4cd192e00c5-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-pz99g\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:37.023692 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:37.023683    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/280fc070-bbb9-44f4-92b4-a4cd192e00c5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-pz99g\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:37.023805 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:37.023767    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/280fc070-bbb9-44f4-92b4-a4cd192e00c5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-pz99g\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:37.035476 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:37.035398    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/280fc070-bbb9-44f4-92b4-a4cd192e00c5-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-pz99g\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:37.039922 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:37.039894    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s58d\" (UniqueName: \"kubernetes.io/projected/280fc070-bbb9-44f4-92b4-a4cd192e00c5-kube-api-access-2s58d\") pod \"istiod-openshift-gateway-7b7995d5cd-pz99g\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:37.093439 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:37.093411    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:37.417333 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:37.417309    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"]
Sep 19 01:17:38.523406 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:17:38.523375    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod280fc070_bbb9_44f4_92b4_a4cd192e00c5.slice/crio-e56b64fb22b79a2eebed1816f3671cd66ddb7da8a32b335c4f4af9a92446983a WatchSource:0}: Error finding container e56b64fb22b79a2eebed1816f3671cd66ddb7da8a32b335c4f4af9a92446983a: Status 404 returned error can't find the container with id e56b64fb22b79a2eebed1816f3671cd66ddb7da8a32b335c4f4af9a92446983a
Sep 19 01:17:38.699931 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:38.699891    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g" event={"ID":"280fc070-bbb9-44f4-92b4-a4cd192e00c5","Type":"ContainerStarted","Data":"e56b64fb22b79a2eebed1816f3671cd66ddb7da8a32b335c4f4af9a92446983a"}
Sep 19 01:17:38.701797 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:38.701760    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c6796f956-wzrsb" event={"ID":"db018a41-4085-448a-bcae-81fe88b9f53a","Type":"ContainerStarted","Data":"14b9fe558b1fc15ed256393d9ece3e1e000a901a6d969c3d80ff39af1952c38c"}
Sep 19 01:17:38.721978 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:38.721920    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5c6796f956-wzrsb" podStartSLOduration=1.7468868739999999 podStartE2EDuration="4.721901555s" totalImagesPullingTime="2.975014681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:17:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:17:38.720636469 +0000 UTC m=+313.627498531" watchObservedRunningTime="2026-09-19 01:17:38.721901555 +0000 UTC m=+313.628763603"
Sep 19 01:17:41.275834 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:41.275794    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 19 01:17:41.276239 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:41.275876    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 19 01:17:41.721370 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:41.721293    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g" event={"ID":"280fc070-bbb9-44f4-92b4-a4cd192e00c5","Type":"ContainerStarted","Data":"73a60ad04dcaa2dc0ed6d51ae7342dda34322027d62dbacf9cf14158cc06d8d7"}
Sep 19 01:17:41.721370 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:41.721368    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:41.743526 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:41.743439    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g" podStartSLOduration=2.9935232899999997 podStartE2EDuration="5.74341644s" totalImagesPullingTime="2.74989315s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:17:41.742376395 +0000 UTC m=+316.649238447" watchObservedRunningTime="2026-09-19 01:17:41.74341644 +0000 UTC m=+316.650278489"
Sep 19 01:17:42.728342 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:42.728310    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:17:46.011225 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:46.011183    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-d9vhq"]
Sep 19 01:17:46.014971 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:46.014948    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-d9vhq"
Sep 19 01:17:46.028050 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:46.028029    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 19 01:17:46.051729 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:46.051707    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-d9vhq"]
Sep 19 01:17:46.104342 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:46.104315    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqbsm\" (UniqueName: \"kubernetes.io/projected/e7192a01-988e-4143-a5bf-37ac7910fcd4-kube-api-access-dqbsm\") pod \"model-serving-api-54d89bd5fc-d9vhq\" (UID: \"e7192a01-988e-4143-a5bf-37ac7910fcd4\") " pod="opendatahub/model-serving-api-54d89bd5fc-d9vhq"
Sep 19 01:17:46.104471 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:46.104349    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7192a01-988e-4143-a5bf-37ac7910fcd4-tls-certs\") pod \"model-serving-api-54d89bd5fc-d9vhq\" (UID: \"e7192a01-988e-4143-a5bf-37ac7910fcd4\") " pod="opendatahub/model-serving-api-54d89bd5fc-d9vhq"
Sep 19 01:17:46.205087 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:46.205057    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7192a01-988e-4143-a5bf-37ac7910fcd4-tls-certs\") pod \"model-serving-api-54d89bd5fc-d9vhq\" (UID: \"e7192a01-988e-4143-a5bf-37ac7910fcd4\") " pod="opendatahub/model-serving-api-54d89bd5fc-d9vhq"
Sep 19 01:17:46.205249 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:46.205152    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqbsm\" (UniqueName: \"kubernetes.io/projected/e7192a01-988e-4143-a5bf-37ac7910fcd4-kube-api-access-dqbsm\") pod \"model-serving-api-54d89bd5fc-d9vhq\" (UID: \"e7192a01-988e-4143-a5bf-37ac7910fcd4\") " pod="opendatahub/model-serving-api-54d89bd5fc-d9vhq"
Sep 19 01:17:46.205249 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:17:46.205216    2580 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 19 01:17:46.205326 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:17:46.205288    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7192a01-988e-4143-a5bf-37ac7910fcd4-tls-certs podName:e7192a01-988e-4143-a5bf-37ac7910fcd4 nodeName:}" failed. No retries permitted until 2026-09-19 01:17:46.705270834 +0000 UTC m=+321.612132864 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/e7192a01-988e-4143-a5bf-37ac7910fcd4-tls-certs") pod "model-serving-api-54d89bd5fc-d9vhq" (UID: "e7192a01-988e-4143-a5bf-37ac7910fcd4") : secret "model-serving-api-tls" not found
Sep 19 01:17:46.213749 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:46.213728    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqbsm\" (UniqueName: \"kubernetes.io/projected/e7192a01-988e-4143-a5bf-37ac7910fcd4-kube-api-access-dqbsm\") pod \"model-serving-api-54d89bd5fc-d9vhq\" (UID: \"e7192a01-988e-4143-a5bf-37ac7910fcd4\") " pod="opendatahub/model-serving-api-54d89bd5fc-d9vhq"
Sep 19 01:17:46.708912 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:46.708883    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7192a01-988e-4143-a5bf-37ac7910fcd4-tls-certs\") pod \"model-serving-api-54d89bd5fc-d9vhq\" (UID: \"e7192a01-988e-4143-a5bf-37ac7910fcd4\") " pod="opendatahub/model-serving-api-54d89bd5fc-d9vhq"
Sep 19 01:17:46.711300 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:46.711279    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7192a01-988e-4143-a5bf-37ac7910fcd4-tls-certs\") pod \"model-serving-api-54d89bd5fc-d9vhq\" (UID: \"e7192a01-988e-4143-a5bf-37ac7910fcd4\") " pod="opendatahub/model-serving-api-54d89bd5fc-d9vhq"
Sep 19 01:17:46.924948 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:46.924917    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-d9vhq"
Sep 19 01:17:47.196076 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:47.196017    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-d9vhq"]
Sep 19 01:17:47.201126 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:17:47.201094    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7192a01_988e_4143_a5bf_37ac7910fcd4.slice/crio-82fa8237f49b955f4e534000317dfae14792e4b79e6320cc57bcb3adafdba2cb WatchSource:0}: Error finding container 82fa8237f49b955f4e534000317dfae14792e4b79e6320cc57bcb3adafdba2cb: Status 404 returned error can't find the container with id 82fa8237f49b955f4e534000317dfae14792e4b79e6320cc57bcb3adafdba2cb
Sep 19 01:17:47.754208 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:47.754174    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-d9vhq" event={"ID":"e7192a01-988e-4143-a5bf-37ac7910fcd4","Type":"ContainerStarted","Data":"82fa8237f49b955f4e534000317dfae14792e4b79e6320cc57bcb3adafdba2cb"}
Sep 19 01:17:49.768726 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:49.768692    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-d9vhq" event={"ID":"e7192a01-988e-4143-a5bf-37ac7910fcd4","Type":"ContainerStarted","Data":"a3097123d30cb8a82863c4c085f240784f34811f9b9bf8b1b41f6aa59e7b373a"}
Sep 19 01:17:49.769208 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:49.768903    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-d9vhq"
Sep 19 01:17:49.788539 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:49.788470    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-d9vhq" podStartSLOduration=3.013303347 podStartE2EDuration="4.788456585s" totalImagesPullingTime="1.775153238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:17:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:17:49.787905981 +0000 UTC m=+324.694768028" watchObservedRunningTime="2026-09-19 01:17:49.788456585 +0000 UTC m=+324.695318632"
Sep 19 01:17:57.139912 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.139868    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-45rlh"]
Sep 19 01:17:57.143983 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.143959    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-45rlh"
Sep 19 01:17:57.149447 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.149427    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-cl6v6"]
Sep 19 01:17:57.152942 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.152926    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cl6v6"
Sep 19 01:17:57.173278 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.173259    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-849rw\""
Sep 19 01:17:57.173837 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.173818    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 19 01:17:57.173915 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.173821    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 19 01:17:57.174219 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.174205    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-nhwxj\""
Sep 19 01:17:57.176528 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.176507    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-45rlh"]
Sep 19 01:17:57.189286 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.189263    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-cl6v6"]
Sep 19 01:17:57.300372 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.300338    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6wfq\" (UniqueName: \"kubernetes.io/projected/1e5626e3-1d2c-470c-b619-17ec423c7922-kube-api-access-c6wfq\") pod \"kserve-controller-manager-d6f4b65b8-45rlh\" (UID: \"1e5626e3-1d2c-470c-b619-17ec423c7922\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-45rlh"
Sep 19 01:17:57.300551 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.300385    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf96s\" (UniqueName: \"kubernetes.io/projected/74af2f71-487e-406c-b594-3292cde8ea44-kube-api-access-gf96s\") pod \"llmisvc-controller-manager-777c6bd7d5-cl6v6\" (UID: \"74af2f71-487e-406c-b594-3292cde8ea44\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cl6v6"
Sep 19 01:17:57.300551 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.300440    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e5626e3-1d2c-470c-b619-17ec423c7922-cert\") pod \"kserve-controller-manager-d6f4b65b8-45rlh\" (UID: \"1e5626e3-1d2c-470c-b619-17ec423c7922\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-45rlh"
Sep 19 01:17:57.300551 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.300455    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74af2f71-487e-406c-b594-3292cde8ea44-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-cl6v6\" (UID: \"74af2f71-487e-406c-b594-3292cde8ea44\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cl6v6"
Sep 19 01:17:57.401517 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.401431    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e5626e3-1d2c-470c-b619-17ec423c7922-cert\") pod \"kserve-controller-manager-d6f4b65b8-45rlh\" (UID: \"1e5626e3-1d2c-470c-b619-17ec423c7922\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-45rlh"
Sep 19 01:17:57.401517 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.401457    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74af2f71-487e-406c-b594-3292cde8ea44-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-cl6v6\" (UID: \"74af2f71-487e-406c-b594-3292cde8ea44\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cl6v6"
Sep 19 01:17:57.401712 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.401527    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6wfq\" (UniqueName: \"kubernetes.io/projected/1e5626e3-1d2c-470c-b619-17ec423c7922-kube-api-access-c6wfq\") pod \"kserve-controller-manager-d6f4b65b8-45rlh\" (UID: \"1e5626e3-1d2c-470c-b619-17ec423c7922\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-45rlh"
Sep 19 01:17:57.401712 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.401562    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf96s\" (UniqueName: \"kubernetes.io/projected/74af2f71-487e-406c-b594-3292cde8ea44-kube-api-access-gf96s\") pod \"llmisvc-controller-manager-777c6bd7d5-cl6v6\" (UID: \"74af2f71-487e-406c-b594-3292cde8ea44\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cl6v6"
Sep 19 01:17:57.401712 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:17:57.401631    2580 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 19 01:17:57.401712 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:17:57.401688    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74af2f71-487e-406c-b594-3292cde8ea44-cert podName:74af2f71-487e-406c-b594-3292cde8ea44 nodeName:}" failed. No retries permitted until 2026-09-19 01:17:57.901674938 +0000 UTC m=+332.808536964 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/74af2f71-487e-406c-b594-3292cde8ea44-cert") pod "llmisvc-controller-manager-777c6bd7d5-cl6v6" (UID: "74af2f71-487e-406c-b594-3292cde8ea44") : secret "llmisvc-webhook-server-cert" not found
Sep 19 01:17:57.403950 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.403928    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e5626e3-1d2c-470c-b619-17ec423c7922-cert\") pod \"kserve-controller-manager-d6f4b65b8-45rlh\" (UID: \"1e5626e3-1d2c-470c-b619-17ec423c7922\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-45rlh"
Sep 19 01:17:57.412304 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.412286    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6wfq\" (UniqueName: \"kubernetes.io/projected/1e5626e3-1d2c-470c-b619-17ec423c7922-kube-api-access-c6wfq\") pod \"kserve-controller-manager-d6f4b65b8-45rlh\" (UID: \"1e5626e3-1d2c-470c-b619-17ec423c7922\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-45rlh"
Sep 19 01:17:57.412386 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.412302    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf96s\" (UniqueName: \"kubernetes.io/projected/74af2f71-487e-406c-b594-3292cde8ea44-kube-api-access-gf96s\") pod \"llmisvc-controller-manager-777c6bd7d5-cl6v6\" (UID: \"74af2f71-487e-406c-b594-3292cde8ea44\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cl6v6"
Sep 19 01:17:57.458314 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.458277    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-45rlh"
Sep 19 01:17:57.744310 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.744275    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-45rlh"]
Sep 19 01:17:57.748461 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:17:57.748435    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e5626e3_1d2c_470c_b619_17ec423c7922.slice/crio-8669b9dc6882e3d0552eb42ec98e310014fceae3d519fcf14abb408e6983a917 WatchSource:0}: Error finding container 8669b9dc6882e3d0552eb42ec98e310014fceae3d519fcf14abb408e6983a917: Status 404 returned error can't find the container with id 8669b9dc6882e3d0552eb42ec98e310014fceae3d519fcf14abb408e6983a917
Sep 19 01:17:57.809026 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.809000    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-45rlh" event={"ID":"1e5626e3-1d2c-470c-b619-17ec423c7922","Type":"ContainerStarted","Data":"8669b9dc6882e3d0552eb42ec98e310014fceae3d519fcf14abb408e6983a917"}
Sep 19 01:17:57.905452 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.905429    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74af2f71-487e-406c-b594-3292cde8ea44-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-cl6v6\" (UID: \"74af2f71-487e-406c-b594-3292cde8ea44\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cl6v6"
Sep 19 01:17:57.907681 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:57.907655    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74af2f71-487e-406c-b594-3292cde8ea44-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-cl6v6\" (UID: \"74af2f71-487e-406c-b594-3292cde8ea44\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cl6v6"
Sep 19 01:17:58.068855 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:58.068786    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cl6v6"
Sep 19 01:17:58.362112 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:58.362084    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-cl6v6"]
Sep 19 01:17:58.363252 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:17:58.363226    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod74af2f71_487e_406c_b594_3292cde8ea44.slice/crio-8717578fe79386e088c71d4a4b1268f529190ea963089c6105dbc061fdd8e6c5 WatchSource:0}: Error finding container 8717578fe79386e088c71d4a4b1268f529190ea963089c6105dbc061fdd8e6c5: Status 404 returned error can't find the container with id 8717578fe79386e088c71d4a4b1268f529190ea963089c6105dbc061fdd8e6c5
Sep 19 01:17:58.815042 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:17:58.815008    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cl6v6" event={"ID":"74af2f71-487e-406c-b594-3292cde8ea44","Type":"ContainerStarted","Data":"8717578fe79386e088c71d4a4b1268f529190ea963089c6105dbc061fdd8e6c5"}
Sep 19 01:18:00.585401 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:00.585373    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"]
Sep 19 01:18:00.585749 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:00.585561    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g" podUID="280fc070-bbb9-44f4-92b4-a4cd192e00c5" containerName="discovery" containerID="cri-o://73a60ad04dcaa2dc0ed6d51ae7342dda34322027d62dbacf9cf14158cc06d8d7" gracePeriod=30
Sep 19 01:18:00.727707 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:00.727667    2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-pz99g container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.35:8080/ready\": dial tcp 10.132.0.35:8080: connect: connection refused" start-of-body=
Sep 19 01:18:00.727844 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:00.727747    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g" podUID="280fc070-bbb9-44f4-92b4-a4cd192e00c5" containerName="discovery" probeResult="failure" output="Get \"http://10.132.0.35:8080/ready\": dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 19 01:18:00.780051 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:00.779716    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-d9vhq"
Sep 19 01:18:00.831197 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:00.831152    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-45rlh" event={"ID":"1e5626e3-1d2c-470c-b619-17ec423c7922","Type":"ContainerStarted","Data":"783199b6c83f5a79213556d8ac3e3c490f3bb6ac56edbde3805b9edce18bb9ee"}
Sep 19 01:18:00.831405 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:00.831223    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-45rlh"
Sep 19 01:18:00.833821 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:00.833793    2580 generic.go:356] "Generic (PLEG): container finished" podID="280fc070-bbb9-44f4-92b4-a4cd192e00c5" containerID="73a60ad04dcaa2dc0ed6d51ae7342dda34322027d62dbacf9cf14158cc06d8d7" exitCode=0
Sep 19 01:18:00.833936 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:00.833822    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g" event={"ID":"280fc070-bbb9-44f4-92b4-a4cd192e00c5","Type":"ContainerDied","Data":"73a60ad04dcaa2dc0ed6d51ae7342dda34322027d62dbacf9cf14158cc06d8d7"}
Sep 19 01:18:00.859367 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:00.859322    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-45rlh" podStartSLOduration=1.294143324 podStartE2EDuration="3.859303668s" totalImagesPullingTime="2.565160344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:18:00.856746667 +0000 UTC m=+335.763608716" watchObservedRunningTime="2026-09-19 01:18:00.859303668 +0000 UTC m=+335.766165716"
Sep 19 01:18:01.372170 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.372145    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:18:01.538710 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.538685    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/280fc070-bbb9-44f4-92b4-a4cd192e00c5-cacerts\") pod \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") "
Sep 19 01:18:01.538820 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.538737    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/280fc070-bbb9-44f4-92b4-a4cd192e00c5-local-certs\") pod \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") "
Sep 19 01:18:01.538820 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.538811    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/280fc070-bbb9-44f4-92b4-a4cd192e00c5-istio-csr-ca-configmap\") pod \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") "
Sep 19 01:18:01.538931 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.538848    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/280fc070-bbb9-44f4-92b4-a4cd192e00c5-istio-token\") pod \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") "
Sep 19 01:18:01.538931 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.538874    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/280fc070-bbb9-44f4-92b4-a4cd192e00c5-istio-kubeconfig\") pod \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") "
Sep 19 01:18:01.538931 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.538914    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/280fc070-bbb9-44f4-92b4-a4cd192e00c5-istio-csr-dns-cert\") pod \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") "
Sep 19 01:18:01.539474 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.538967    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2s58d\" (UniqueName: \"kubernetes.io/projected/280fc070-bbb9-44f4-92b4-a4cd192e00c5-kube-api-access-2s58d\") pod \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\" (UID: \"280fc070-bbb9-44f4-92b4-a4cd192e00c5\") "
Sep 19 01:18:01.539707 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.539678    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/280fc070-bbb9-44f4-92b4-a4cd192e00c5-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "280fc070-bbb9-44f4-92b4-a4cd192e00c5" (UID: "280fc070-bbb9-44f4-92b4-a4cd192e00c5"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:18:01.542022 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.541931    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/280fc070-bbb9-44f4-92b4-a4cd192e00c5-istio-token" (OuterVolumeSpecName: "istio-token") pod "280fc070-bbb9-44f4-92b4-a4cd192e00c5" (UID: "280fc070-bbb9-44f4-92b4-a4cd192e00c5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:18:01.542022 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.541986    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/280fc070-bbb9-44f4-92b4-a4cd192e00c5-cacerts" (OuterVolumeSpecName: "cacerts") pod "280fc070-bbb9-44f4-92b4-a4cd192e00c5" (UID: "280fc070-bbb9-44f4-92b4-a4cd192e00c5"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:18:01.542425 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.542393    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/280fc070-bbb9-44f4-92b4-a4cd192e00c5-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "280fc070-bbb9-44f4-92b4-a4cd192e00c5" (UID: "280fc070-bbb9-44f4-92b4-a4cd192e00c5"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:18:01.542630 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.542597    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/280fc070-bbb9-44f4-92b4-a4cd192e00c5-local-certs" (OuterVolumeSpecName: "local-certs") pod "280fc070-bbb9-44f4-92b4-a4cd192e00c5" (UID: "280fc070-bbb9-44f4-92b4-a4cd192e00c5"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:18:01.542713 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.542666    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/280fc070-bbb9-44f4-92b4-a4cd192e00c5-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "280fc070-bbb9-44f4-92b4-a4cd192e00c5" (UID: "280fc070-bbb9-44f4-92b4-a4cd192e00c5"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:18:01.543349 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.543331    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/280fc070-bbb9-44f4-92b4-a4cd192e00c5-kube-api-access-2s58d" (OuterVolumeSpecName: "kube-api-access-2s58d") pod "280fc070-bbb9-44f4-92b4-a4cd192e00c5" (UID: "280fc070-bbb9-44f4-92b4-a4cd192e00c5"). InnerVolumeSpecName "kube-api-access-2s58d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:18:01.640604 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.640538    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/280fc070-bbb9-44f4-92b4-a4cd192e00c5-istio-token\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:18:01.640604 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.640562    2580 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/280fc070-bbb9-44f4-92b4-a4cd192e00c5-istio-kubeconfig\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:18:01.640604 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.640571    2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/280fc070-bbb9-44f4-92b4-a4cd192e00c5-istio-csr-dns-cert\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:18:01.640604 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.640579    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2s58d\" (UniqueName: \"kubernetes.io/projected/280fc070-bbb9-44f4-92b4-a4cd192e00c5-kube-api-access-2s58d\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:18:01.640604 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.640588    2580 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/280fc070-bbb9-44f4-92b4-a4cd192e00c5-cacerts\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:18:01.640604 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.640597    2580 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/280fc070-bbb9-44f4-92b4-a4cd192e00c5-local-certs\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:18:01.640604 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.640606    2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/280fc070-bbb9-44f4-92b4-a4cd192e00c5-istio-csr-ca-configmap\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:18:01.840266 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.840237    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"
Sep 19 01:18:01.840441 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.840285    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g" event={"ID":"280fc070-bbb9-44f4-92b4-a4cd192e00c5","Type":"ContainerDied","Data":"e56b64fb22b79a2eebed1816f3671cd66ddb7da8a32b335c4f4af9a92446983a"}
Sep 19 01:18:01.840441 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.840347    2580 scope.go:117] "RemoveContainer" containerID="73a60ad04dcaa2dc0ed6d51ae7342dda34322027d62dbacf9cf14158cc06d8d7"
Sep 19 01:18:01.842133 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.842101    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cl6v6" event={"ID":"74af2f71-487e-406c-b594-3292cde8ea44","Type":"ContainerStarted","Data":"4ac0fbebc4caaaa7438d048c34ebc2eb77d75df4d8b3ac3c356e59739869b70a"}
Sep 19 01:18:01.842309 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.842290    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cl6v6"
Sep 19 01:18:01.863526 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.863469    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cl6v6" podStartSLOduration=1.825242245 podStartE2EDuration="4.863455902s" totalImagesPullingTime="3.038213657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:18:01.861380247 +0000 UTC m=+336.768242294" watchObservedRunningTime="2026-09-19 01:18:01.863455902 +0000 UTC m=+336.770317950"
Sep 19 01:18:01.880819 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.880795    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"]
Sep 19 01:18:01.883845 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:01.883824    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pz99g"]
Sep 19 01:18:03.063195 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:03.063163    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="280fc070-bbb9-44f4-92b4-a4cd192e00c5" path="/var/lib/kubelet/pods/280fc070-bbb9-44f4-92b4-a4cd192e00c5/volumes"
Sep 19 01:18:31.847477 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:31.847442    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-45rlh"
Sep 19 01:18:32.851651 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:32.851618    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cl6v6"
Sep 19 01:18:51.563209 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.563171    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8"]
Sep 19 01:18:51.563639 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.563551    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="280fc070-bbb9-44f4-92b4-a4cd192e00c5" containerName="discovery"
Sep 19 01:18:51.563639 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.563564    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="280fc070-bbb9-44f4-92b4-a4cd192e00c5" containerName="discovery"
Sep 19 01:18:51.563639 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.563630    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="280fc070-bbb9-44f4-92b4-a4cd192e00c5" containerName="discovery"
Sep 19 01:18:51.566882 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.566865    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8"
Sep 19 01:18:51.569209 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.569187    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kfj97\""
Sep 19 01:18:51.569209 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.569197    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 19 01:18:51.569876 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.569860    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 19 01:18:51.575990 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.575968    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8"]
Sep 19 01:18:51.665724 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.665691    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c4f0f08-8b4a-49fc-8a34-c80e89209690-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8\" (UID: \"2c4f0f08-8b4a-49fc-8a34-c80e89209690\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8"
Sep 19 01:18:51.665892 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.665732    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vdxp\" (UniqueName: \"kubernetes.io/projected/2c4f0f08-8b4a-49fc-8a34-c80e89209690-kube-api-access-8vdxp\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8\" (UID: \"2c4f0f08-8b4a-49fc-8a34-c80e89209690\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8"
Sep 19 01:18:51.665892 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.665755    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c4f0f08-8b4a-49fc-8a34-c80e89209690-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8\" (UID: \"2c4f0f08-8b4a-49fc-8a34-c80e89209690\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8"
Sep 19 01:18:51.670246 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.670208    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l"]
Sep 19 01:18:51.673839 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.673824    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l"
Sep 19 01:18:51.685906 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.685878    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l"]
Sep 19 01:18:51.760788 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.760760    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n"]
Sep 19 01:18:51.764433 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.764417    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n"
Sep 19 01:18:51.766992 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.766962    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1029fd0f-2726-41b2-b2ba-f3314f64615d-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l\" (UID: \"1029fd0f-2726-41b2-b2ba-f3314f64615d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l"
Sep 19 01:18:51.766992 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.766990    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1029fd0f-2726-41b2-b2ba-f3314f64615d-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l\" (UID: \"1029fd0f-2726-41b2-b2ba-f3314f64615d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l"
Sep 19 01:18:51.767169 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.767013    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c4f0f08-8b4a-49fc-8a34-c80e89209690-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8\" (UID: \"2c4f0f08-8b4a-49fc-8a34-c80e89209690\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8"
Sep 19 01:18:51.767169 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.767031    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxs2m\" (UniqueName: \"kubernetes.io/projected/1029fd0f-2726-41b2-b2ba-f3314f64615d-kube-api-access-xxs2m\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l\" (UID: \"1029fd0f-2726-41b2-b2ba-f3314f64615d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l"
Sep 19 01:18:51.767169 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.767065    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vdxp\" (UniqueName: \"kubernetes.io/projected/2c4f0f08-8b4a-49fc-8a34-c80e89209690-kube-api-access-8vdxp\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8\" (UID: \"2c4f0f08-8b4a-49fc-8a34-c80e89209690\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8"
Sep 19 01:18:51.767169 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.767121    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c4f0f08-8b4a-49fc-8a34-c80e89209690-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8\" (UID: \"2c4f0f08-8b4a-49fc-8a34-c80e89209690\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8"
Sep 19 01:18:51.767169 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.767161    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92df7e56-8584-4e44-a365-4fd75758e912-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n\" (UID: \"92df7e56-8584-4e44-a365-4fd75758e912\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n"
Sep 19 01:18:51.767371 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.767185    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92df7e56-8584-4e44-a365-4fd75758e912-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n\" (UID: \"92df7e56-8584-4e44-a365-4fd75758e912\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n"
Sep 19 01:18:51.767371 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.767204    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcd56\" (UniqueName: \"kubernetes.io/projected/92df7e56-8584-4e44-a365-4fd75758e912-kube-api-access-lcd56\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n\" (UID: \"92df7e56-8584-4e44-a365-4fd75758e912\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n"
Sep 19 01:18:51.767371 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.767345    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c4f0f08-8b4a-49fc-8a34-c80e89209690-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8\" (UID: \"2c4f0f08-8b4a-49fc-8a34-c80e89209690\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8"
Sep 19 01:18:51.767510 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.767481    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c4f0f08-8b4a-49fc-8a34-c80e89209690-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8\" (UID: \"2c4f0f08-8b4a-49fc-8a34-c80e89209690\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8"
Sep 19 01:18:51.777384 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.777364    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n"]
Sep 19 01:18:51.780093 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.780065    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vdxp\" (UniqueName: \"kubernetes.io/projected/2c4f0f08-8b4a-49fc-8a34-c80e89209690-kube-api-access-8vdxp\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8\" (UID: \"2c4f0f08-8b4a-49fc-8a34-c80e89209690\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8"
Sep 19 01:18:51.868071 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.867990    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxs2m\" (UniqueName: \"kubernetes.io/projected/1029fd0f-2726-41b2-b2ba-f3314f64615d-kube-api-access-xxs2m\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l\" (UID: \"1029fd0f-2726-41b2-b2ba-f3314f64615d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l"
Sep 19 01:18:51.868071 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.868045    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92df7e56-8584-4e44-a365-4fd75758e912-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n\" (UID: \"92df7e56-8584-4e44-a365-4fd75758e912\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n"
Sep 19 01:18:51.868071 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.868065    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92df7e56-8584-4e44-a365-4fd75758e912-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n\" (UID: \"92df7e56-8584-4e44-a365-4fd75758e912\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n"
Sep 19 01:18:51.868340 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.868090    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcd56\" (UniqueName: \"kubernetes.io/projected/92df7e56-8584-4e44-a365-4fd75758e912-kube-api-access-lcd56\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n\" (UID: \"92df7e56-8584-4e44-a365-4fd75758e912\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n"
Sep 19 01:18:51.868340 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.868158    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1029fd0f-2726-41b2-b2ba-f3314f64615d-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l\" (UID: \"1029fd0f-2726-41b2-b2ba-f3314f64615d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l"
Sep 19 01:18:51.868340 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.868180    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1029fd0f-2726-41b2-b2ba-f3314f64615d-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l\" (UID: \"1029fd0f-2726-41b2-b2ba-f3314f64615d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l"
Sep 19 01:18:51.868589 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.868571    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92df7e56-8584-4e44-a365-4fd75758e912-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n\" (UID: \"92df7e56-8584-4e44-a365-4fd75758e912\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n"
Sep 19 01:18:51.868637 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.868576    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1029fd0f-2726-41b2-b2ba-f3314f64615d-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l\" (UID: \"1029fd0f-2726-41b2-b2ba-f3314f64615d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l"
Sep 19 01:18:51.868637 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.868602    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92df7e56-8584-4e44-a365-4fd75758e912-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n\" (UID: \"92df7e56-8584-4e44-a365-4fd75758e912\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n"
Sep 19 01:18:51.868637 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.868602    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1029fd0f-2726-41b2-b2ba-f3314f64615d-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l\" (UID: \"1029fd0f-2726-41b2-b2ba-f3314f64615d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l"
Sep 19 01:18:51.877860 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.877826    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8"
Sep 19 01:18:51.885767 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.885748    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxs2m\" (UniqueName: \"kubernetes.io/projected/1029fd0f-2726-41b2-b2ba-f3314f64615d-kube-api-access-xxs2m\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l\" (UID: \"1029fd0f-2726-41b2-b2ba-f3314f64615d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l"
Sep 19 01:18:51.888067 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.888044    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcd56\" (UniqueName: \"kubernetes.io/projected/92df7e56-8584-4e44-a365-4fd75758e912-kube-api-access-lcd56\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n\" (UID: \"92df7e56-8584-4e44-a365-4fd75758e912\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n"
Sep 19 01:18:51.952195 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.952165    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv"]
Sep 19 01:18:51.957096 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.957076    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv"
Sep 19 01:18:51.969207 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.969187    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6f77707-1c65-4c81-be7f-0178cf865f20-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv\" (UID: \"b6f77707-1c65-4c81-be7f-0178cf865f20\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv"
Sep 19 01:18:51.969310 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.969251    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tpl7\" (UniqueName: \"kubernetes.io/projected/b6f77707-1c65-4c81-be7f-0178cf865f20-kube-api-access-8tpl7\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv\" (UID: \"b6f77707-1c65-4c81-be7f-0178cf865f20\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv"
Sep 19 01:18:51.969310 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.969290    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6f77707-1c65-4c81-be7f-0178cf865f20-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv\" (UID: \"b6f77707-1c65-4c81-be7f-0178cf865f20\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv"
Sep 19 01:18:51.974387 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.974367    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv"]
Sep 19 01:18:51.984797 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:51.984778    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l"
Sep 19 01:18:52.070477 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:52.070443    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tpl7\" (UniqueName: \"kubernetes.io/projected/b6f77707-1c65-4c81-be7f-0178cf865f20-kube-api-access-8tpl7\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv\" (UID: \"b6f77707-1c65-4c81-be7f-0178cf865f20\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv"
Sep 19 01:18:52.070635 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:52.070514    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6f77707-1c65-4c81-be7f-0178cf865f20-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv\" (UID: \"b6f77707-1c65-4c81-be7f-0178cf865f20\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv"
Sep 19 01:18:52.070635 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:52.070584    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6f77707-1c65-4c81-be7f-0178cf865f20-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv\" (UID: \"b6f77707-1c65-4c81-be7f-0178cf865f20\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv"
Sep 19 01:18:52.070974 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:52.070951    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6f77707-1c65-4c81-be7f-0178cf865f20-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv\" (UID: \"b6f77707-1c65-4c81-be7f-0178cf865f20\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv"
Sep 19 01:18:52.071043 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:52.071003    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6f77707-1c65-4c81-be7f-0178cf865f20-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv\" (UID: \"b6f77707-1c65-4c81-be7f-0178cf865f20\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv"
Sep 19 01:18:52.073360 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:52.073341    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n"
Sep 19 01:18:52.087450 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:52.087430    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tpl7\" (UniqueName: \"kubernetes.io/projected/b6f77707-1c65-4c81-be7f-0178cf865f20-kube-api-access-8tpl7\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv\" (UID: \"b6f77707-1c65-4c81-be7f-0178cf865f20\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv"
Sep 19 01:18:52.161970 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:52.161943    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8"]
Sep 19 01:18:52.165228 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:18:52.165198    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c4f0f08_8b4a_49fc_8a34_c80e89209690.slice/crio-b564700ad9ffc877b3e0d28a78fd83cd17ddf21784310688ef355ed515b5dd94 WatchSource:0}: Error finding container b564700ad9ffc877b3e0d28a78fd83cd17ddf21784310688ef355ed515b5dd94: Status 404 returned error can't find the container with id b564700ad9ffc877b3e0d28a78fd83cd17ddf21784310688ef355ed515b5dd94
Sep 19 01:18:52.261484 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:52.261459    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l"]
Sep 19 01:18:52.263974 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:18:52.263940    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1029fd0f_2726_41b2_b2ba_f3314f64615d.slice/crio-3c036acf871e4e7c1b650e37bc9622d291efdeb5ae96db5fa520af94f06423f8 WatchSource:0}: Error finding container 3c036acf871e4e7c1b650e37bc9622d291efdeb5ae96db5fa520af94f06423f8: Status 404 returned error can't find the container with id 3c036acf871e4e7c1b650e37bc9622d291efdeb5ae96db5fa520af94f06423f8
Sep 19 01:18:52.266044 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:52.266022    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv"
Sep 19 01:18:52.365757 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:52.365729    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n"]
Sep 19 01:18:52.367951 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:18:52.367927    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92df7e56_8584_4e44_a365_4fd75758e912.slice/crio-b1235f8aea62faa30e937563c63acaf6d68a3b2b80d1b48699bc3b342e985aa8 WatchSource:0}: Error finding container b1235f8aea62faa30e937563c63acaf6d68a3b2b80d1b48699bc3b342e985aa8: Status 404 returned error can't find the container with id b1235f8aea62faa30e937563c63acaf6d68a3b2b80d1b48699bc3b342e985aa8
Sep 19 01:18:52.546946 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:52.546917    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv"]
Sep 19 01:18:52.576285 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:18:52.576255    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6f77707_1c65_4c81_be7f_0178cf865f20.slice/crio-5ff6c75a49eae782ae0cd5998d6ff91069c3e1494fc3b501b55af9bc10cb5224 WatchSource:0}: Error finding container 5ff6c75a49eae782ae0cd5998d6ff91069c3e1494fc3b501b55af9bc10cb5224: Status 404 returned error can't find the container with id 5ff6c75a49eae782ae0cd5998d6ff91069c3e1494fc3b501b55af9bc10cb5224
Sep 19 01:18:53.101910 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:53.101877    2580 generic.go:356] "Generic (PLEG): container finished" podID="2c4f0f08-8b4a-49fc-8a34-c80e89209690" containerID="e9adbdecacac478574e754b3b5744c6fa719463f3d17ea89ba25c723271a4cab" exitCode=0
Sep 19 01:18:53.102099 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:53.101967    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8" event={"ID":"2c4f0f08-8b4a-49fc-8a34-c80e89209690","Type":"ContainerDied","Data":"e9adbdecacac478574e754b3b5744c6fa719463f3d17ea89ba25c723271a4cab"}
Sep 19 01:18:53.102099 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:53.102016    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8" event={"ID":"2c4f0f08-8b4a-49fc-8a34-c80e89209690","Type":"ContainerStarted","Data":"b564700ad9ffc877b3e0d28a78fd83cd17ddf21784310688ef355ed515b5dd94"}
Sep 19 01:18:53.103488 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:53.103465    2580 generic.go:356] "Generic (PLEG): container finished" podID="1029fd0f-2726-41b2-b2ba-f3314f64615d" containerID="6a950a4b2633f3ab5233b333b8fb8d9977e6d78905e87330476babb79dd55125" exitCode=0
Sep 19 01:18:53.103581 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:53.103528    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l" event={"ID":"1029fd0f-2726-41b2-b2ba-f3314f64615d","Type":"ContainerDied","Data":"6a950a4b2633f3ab5233b333b8fb8d9977e6d78905e87330476babb79dd55125"}
Sep 19 01:18:53.103642 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:53.103586    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l" event={"ID":"1029fd0f-2726-41b2-b2ba-f3314f64615d","Type":"ContainerStarted","Data":"3c036acf871e4e7c1b650e37bc9622d291efdeb5ae96db5fa520af94f06423f8"}
Sep 19 01:18:53.105190 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:53.105175    2580 generic.go:356] "Generic (PLEG): container finished" podID="b6f77707-1c65-4c81-be7f-0178cf865f20" containerID="15a7f1670692e5ff7da940bef382c3d14d7006dc0bc8538adb248c0f444736d6" exitCode=0
Sep 19 01:18:53.105279 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:53.105259    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv" event={"ID":"b6f77707-1c65-4c81-be7f-0178cf865f20","Type":"ContainerDied","Data":"15a7f1670692e5ff7da940bef382c3d14d7006dc0bc8538adb248c0f444736d6"}
Sep 19 01:18:53.105335 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:53.105321    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv" event={"ID":"b6f77707-1c65-4c81-be7f-0178cf865f20","Type":"ContainerStarted","Data":"5ff6c75a49eae782ae0cd5998d6ff91069c3e1494fc3b501b55af9bc10cb5224"}
Sep 19 01:18:53.107062 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:53.107044    2580 generic.go:356] "Generic (PLEG): container finished" podID="92df7e56-8584-4e44-a365-4fd75758e912" containerID="c2ab45157ecb83e53067ef2884ed2c7db8b600ee4f656ebb248854cd1015f7f1" exitCode=0
Sep 19 01:18:53.107173 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:53.107101    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n" event={"ID":"92df7e56-8584-4e44-a365-4fd75758e912","Type":"ContainerDied","Data":"c2ab45157ecb83e53067ef2884ed2c7db8b600ee4f656ebb248854cd1015f7f1"}
Sep 19 01:18:53.107173 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:53.107127    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n" event={"ID":"92df7e56-8584-4e44-a365-4fd75758e912","Type":"ContainerStarted","Data":"b1235f8aea62faa30e937563c63acaf6d68a3b2b80d1b48699bc3b342e985aa8"}
Sep 19 01:18:54.114199 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:54.114128    2580 generic.go:356] "Generic (PLEG): container finished" podID="2c4f0f08-8b4a-49fc-8a34-c80e89209690" containerID="81f945661c285fe2796d6ba52a9edd6fd12ea2aabcdddc7813b41ec15382f2a4" exitCode=0
Sep 19 01:18:54.114199 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:54.114178    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8" event={"ID":"2c4f0f08-8b4a-49fc-8a34-c80e89209690","Type":"ContainerDied","Data":"81f945661c285fe2796d6ba52a9edd6fd12ea2aabcdddc7813b41ec15382f2a4"}
Sep 19 01:18:55.121935 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:55.121845    2580 generic.go:356] "Generic (PLEG): container finished" podID="2c4f0f08-8b4a-49fc-8a34-c80e89209690" containerID="e65d1a8714bdecd05fc6080b1e3583012ed9088adb087929023ab8d0fdb6c2c6" exitCode=0
Sep 19 01:18:55.121935 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:55.121920    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8" event={"ID":"2c4f0f08-8b4a-49fc-8a34-c80e89209690","Type":"ContainerDied","Data":"e65d1a8714bdecd05fc6080b1e3583012ed9088adb087929023ab8d0fdb6c2c6"}
Sep 19 01:18:55.123773 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:55.123749    2580 generic.go:356] "Generic (PLEG): container finished" podID="1029fd0f-2726-41b2-b2ba-f3314f64615d" containerID="5dcdc300a617def7e0c5fb2fbbd3039b2c7ef70bdf8b93ea5361951f3bf61428" exitCode=0
Sep 19 01:18:55.123913 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:55.123825    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l" event={"ID":"1029fd0f-2726-41b2-b2ba-f3314f64615d","Type":"ContainerDied","Data":"5dcdc300a617def7e0c5fb2fbbd3039b2c7ef70bdf8b93ea5361951f3bf61428"}
Sep 19 01:18:55.125824 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:55.125799    2580 generic.go:356] "Generic (PLEG): container finished" podID="b6f77707-1c65-4c81-be7f-0178cf865f20" containerID="2b7ddb95d350e79c79c75e21b24ad2633efb0642d4ea36839de8bd124420cee4" exitCode=0
Sep 19 01:18:55.125948 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:55.125824    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv" event={"ID":"b6f77707-1c65-4c81-be7f-0178cf865f20","Type":"ContainerDied","Data":"2b7ddb95d350e79c79c75e21b24ad2633efb0642d4ea36839de8bd124420cee4"}
Sep 19 01:18:55.127826 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:55.127809    2580 generic.go:356] "Generic (PLEG): container finished" podID="92df7e56-8584-4e44-a365-4fd75758e912" containerID="b7acc8c4fa26776697da8750d9b785935441c96aa5e7ec84254c0216d5864739" exitCode=0
Sep 19 01:18:55.127915 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:55.127861    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n" event={"ID":"92df7e56-8584-4e44-a365-4fd75758e912","Type":"ContainerDied","Data":"b7acc8c4fa26776697da8750d9b785935441c96aa5e7ec84254c0216d5864739"}
Sep 19 01:18:56.139357 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:56.139326    2580 generic.go:356] "Generic (PLEG): container finished" podID="1029fd0f-2726-41b2-b2ba-f3314f64615d" containerID="cba45a17c9116f094321e2c4bd78d0bad97189707828b04aa8fb1dabfdbd9b79" exitCode=0
Sep 19 01:18:56.139764 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:56.139409    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l" event={"ID":"1029fd0f-2726-41b2-b2ba-f3314f64615d","Type":"ContainerDied","Data":"cba45a17c9116f094321e2c4bd78d0bad97189707828b04aa8fb1dabfdbd9b79"}
Sep 19 01:18:56.141766 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:56.141738    2580 generic.go:356] "Generic (PLEG): container finished" podID="b6f77707-1c65-4c81-be7f-0178cf865f20" containerID="1dfadf05175a237e4f958fed8e30d03b700b8d639e84123fe4569969ed961638" exitCode=0
Sep 19 01:18:56.141886 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:56.141822    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv" event={"ID":"b6f77707-1c65-4c81-be7f-0178cf865f20","Type":"ContainerDied","Data":"1dfadf05175a237e4f958fed8e30d03b700b8d639e84123fe4569969ed961638"}
Sep 19 01:18:56.143969 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:56.143946    2580 generic.go:356] "Generic (PLEG): container finished" podID="92df7e56-8584-4e44-a365-4fd75758e912" containerID="946609e700fa64d082407fd84116b09c173bd8ff14cbf468ffca75d53a9942e6" exitCode=0
Sep 19 01:18:56.144103 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:56.144025    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n" event={"ID":"92df7e56-8584-4e44-a365-4fd75758e912","Type":"ContainerDied","Data":"946609e700fa64d082407fd84116b09c173bd8ff14cbf468ffca75d53a9942e6"}
Sep 19 01:18:56.275861 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:56.275838    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8"
Sep 19 01:18:56.303741 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:56.303715    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c4f0f08-8b4a-49fc-8a34-c80e89209690-util\") pod \"2c4f0f08-8b4a-49fc-8a34-c80e89209690\" (UID: \"2c4f0f08-8b4a-49fc-8a34-c80e89209690\") "
Sep 19 01:18:56.303890 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:56.303770    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c4f0f08-8b4a-49fc-8a34-c80e89209690-bundle\") pod \"2c4f0f08-8b4a-49fc-8a34-c80e89209690\" (UID: \"2c4f0f08-8b4a-49fc-8a34-c80e89209690\") "
Sep 19 01:18:56.303890 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:56.303804    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vdxp\" (UniqueName: \"kubernetes.io/projected/2c4f0f08-8b4a-49fc-8a34-c80e89209690-kube-api-access-8vdxp\") pod \"2c4f0f08-8b4a-49fc-8a34-c80e89209690\" (UID: \"2c4f0f08-8b4a-49fc-8a34-c80e89209690\") "
Sep 19 01:18:56.304225 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:56.304200    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c4f0f08-8b4a-49fc-8a34-c80e89209690-bundle" (OuterVolumeSpecName: "bundle") pod "2c4f0f08-8b4a-49fc-8a34-c80e89209690" (UID: "2c4f0f08-8b4a-49fc-8a34-c80e89209690"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:18:56.305774 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:56.305754    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c4f0f08-8b4a-49fc-8a34-c80e89209690-kube-api-access-8vdxp" (OuterVolumeSpecName: "kube-api-access-8vdxp") pod "2c4f0f08-8b4a-49fc-8a34-c80e89209690" (UID: "2c4f0f08-8b4a-49fc-8a34-c80e89209690"). InnerVolumeSpecName "kube-api-access-8vdxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:18:56.309372 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:56.309350    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c4f0f08-8b4a-49fc-8a34-c80e89209690-util" (OuterVolumeSpecName: "util") pod "2c4f0f08-8b4a-49fc-8a34-c80e89209690" (UID: "2c4f0f08-8b4a-49fc-8a34-c80e89209690"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:18:56.404568 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:56.404464    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c4f0f08-8b4a-49fc-8a34-c80e89209690-util\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:18:56.404568 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:56.404489    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c4f0f08-8b4a-49fc-8a34-c80e89209690-bundle\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:18:56.404568 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:56.404525    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vdxp\" (UniqueName: \"kubernetes.io/projected/2c4f0f08-8b4a-49fc-8a34-c80e89209690-kube-api-access-8vdxp\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:18:57.152143 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.152116    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8"
Sep 19 01:18:57.152617 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.152194    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jv2x8" event={"ID":"2c4f0f08-8b4a-49fc-8a34-c80e89209690","Type":"ContainerDied","Data":"b564700ad9ffc877b3e0d28a78fd83cd17ddf21784310688ef355ed515b5dd94"}
Sep 19 01:18:57.152617 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.152242    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b564700ad9ffc877b3e0d28a78fd83cd17ddf21784310688ef355ed515b5dd94"
Sep 19 01:18:57.301057 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.301025    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n"
Sep 19 01:18:57.309577 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.309553    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92df7e56-8584-4e44-a365-4fd75758e912-util\") pod \"92df7e56-8584-4e44-a365-4fd75758e912\" (UID: \"92df7e56-8584-4e44-a365-4fd75758e912\") "
Sep 19 01:18:57.309711 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.309591    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92df7e56-8584-4e44-a365-4fd75758e912-bundle\") pod \"92df7e56-8584-4e44-a365-4fd75758e912\" (UID: \"92df7e56-8584-4e44-a365-4fd75758e912\") "
Sep 19 01:18:57.309711 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.309611    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcd56\" (UniqueName: \"kubernetes.io/projected/92df7e56-8584-4e44-a365-4fd75758e912-kube-api-access-lcd56\") pod \"92df7e56-8584-4e44-a365-4fd75758e912\" (UID: \"92df7e56-8584-4e44-a365-4fd75758e912\") "
Sep 19 01:18:57.310100 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.310074    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92df7e56-8584-4e44-a365-4fd75758e912-bundle" (OuterVolumeSpecName: "bundle") pod "92df7e56-8584-4e44-a365-4fd75758e912" (UID: "92df7e56-8584-4e44-a365-4fd75758e912"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:18:57.313022 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.312993    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92df7e56-8584-4e44-a365-4fd75758e912-kube-api-access-lcd56" (OuterVolumeSpecName: "kube-api-access-lcd56") pod "92df7e56-8584-4e44-a365-4fd75758e912" (UID: "92df7e56-8584-4e44-a365-4fd75758e912"). InnerVolumeSpecName "kube-api-access-lcd56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:18:57.315046 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.315026    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92df7e56-8584-4e44-a365-4fd75758e912-util" (OuterVolumeSpecName: "util") pod "92df7e56-8584-4e44-a365-4fd75758e912" (UID: "92df7e56-8584-4e44-a365-4fd75758e912"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:18:57.345044 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.345017    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l"
Sep 19 01:18:57.348093 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.348070    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv"
Sep 19 01:18:57.410353 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.410322    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1029fd0f-2726-41b2-b2ba-f3314f64615d-util\") pod \"1029fd0f-2726-41b2-b2ba-f3314f64615d\" (UID: \"1029fd0f-2726-41b2-b2ba-f3314f64615d\") "
Sep 19 01:18:57.410353 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.410357    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxs2m\" (UniqueName: \"kubernetes.io/projected/1029fd0f-2726-41b2-b2ba-f3314f64615d-kube-api-access-xxs2m\") pod \"1029fd0f-2726-41b2-b2ba-f3314f64615d\" (UID: \"1029fd0f-2726-41b2-b2ba-f3314f64615d\") "
Sep 19 01:18:57.410620 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.410386    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6f77707-1c65-4c81-be7f-0178cf865f20-bundle\") pod \"b6f77707-1c65-4c81-be7f-0178cf865f20\" (UID: \"b6f77707-1c65-4c81-be7f-0178cf865f20\") "
Sep 19 01:18:57.410620 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.410417    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6f77707-1c65-4c81-be7f-0178cf865f20-util\") pod \"b6f77707-1c65-4c81-be7f-0178cf865f20\" (UID: \"b6f77707-1c65-4c81-be7f-0178cf865f20\") "
Sep 19 01:18:57.410620 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.410451    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8tpl7\" (UniqueName: \"kubernetes.io/projected/b6f77707-1c65-4c81-be7f-0178cf865f20-kube-api-access-8tpl7\") pod \"b6f77707-1c65-4c81-be7f-0178cf865f20\" (UID: \"b6f77707-1c65-4c81-be7f-0178cf865f20\") "
Sep 19 01:18:57.410620 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.410470    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1029fd0f-2726-41b2-b2ba-f3314f64615d-bundle\") pod \"1029fd0f-2726-41b2-b2ba-f3314f64615d\" (UID: \"1029fd0f-2726-41b2-b2ba-f3314f64615d\") "
Sep 19 01:18:57.410815 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.410724    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92df7e56-8584-4e44-a365-4fd75758e912-util\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:18:57.410815 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.410746    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92df7e56-8584-4e44-a365-4fd75758e912-bundle\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:18:57.410815 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.410759    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcd56\" (UniqueName: \"kubernetes.io/projected/92df7e56-8584-4e44-a365-4fd75758e912-kube-api-access-lcd56\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:18:57.411286 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.411233    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6f77707-1c65-4c81-be7f-0178cf865f20-bundle" (OuterVolumeSpecName: "bundle") pod "b6f77707-1c65-4c81-be7f-0178cf865f20" (UID: "b6f77707-1c65-4c81-be7f-0178cf865f20"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:18:57.411431 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.411412    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1029fd0f-2726-41b2-b2ba-f3314f64615d-bundle" (OuterVolumeSpecName: "bundle") pod "1029fd0f-2726-41b2-b2ba-f3314f64615d" (UID: "1029fd0f-2726-41b2-b2ba-f3314f64615d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:18:57.413112 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.413083    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1029fd0f-2726-41b2-b2ba-f3314f64615d-kube-api-access-xxs2m" (OuterVolumeSpecName: "kube-api-access-xxs2m") pod "1029fd0f-2726-41b2-b2ba-f3314f64615d" (UID: "1029fd0f-2726-41b2-b2ba-f3314f64615d"). InnerVolumeSpecName "kube-api-access-xxs2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:18:57.413427 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.413408    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6f77707-1c65-4c81-be7f-0178cf865f20-kube-api-access-8tpl7" (OuterVolumeSpecName: "kube-api-access-8tpl7") pod "b6f77707-1c65-4c81-be7f-0178cf865f20" (UID: "b6f77707-1c65-4c81-be7f-0178cf865f20"). InnerVolumeSpecName "kube-api-access-8tpl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:18:57.416604 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.416575    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1029fd0f-2726-41b2-b2ba-f3314f64615d-util" (OuterVolumeSpecName: "util") pod "1029fd0f-2726-41b2-b2ba-f3314f64615d" (UID: "1029fd0f-2726-41b2-b2ba-f3314f64615d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:18:57.420394 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.420355    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6f77707-1c65-4c81-be7f-0178cf865f20-util" (OuterVolumeSpecName: "util") pod "b6f77707-1c65-4c81-be7f-0178cf865f20" (UID: "b6f77707-1c65-4c81-be7f-0178cf865f20"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:18:57.511453 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.511426    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tpl7\" (UniqueName: \"kubernetes.io/projected/b6f77707-1c65-4c81-be7f-0178cf865f20-kube-api-access-8tpl7\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:18:57.511453 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.511449    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1029fd0f-2726-41b2-b2ba-f3314f64615d-bundle\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:18:57.511453 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.511458    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1029fd0f-2726-41b2-b2ba-f3314f64615d-util\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:18:57.511649 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.511466    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxs2m\" (UniqueName: \"kubernetes.io/projected/1029fd0f-2726-41b2-b2ba-f3314f64615d-kube-api-access-xxs2m\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:18:57.511649 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.511475    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6f77707-1c65-4c81-be7f-0178cf865f20-bundle\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:18:57.511649 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:57.511482    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6f77707-1c65-4c81-be7f-0178cf865f20-util\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:18:58.160301 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:58.160275    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l"
Sep 19 01:18:58.160995 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:58.160275    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a5655l" event={"ID":"1029fd0f-2726-41b2-b2ba-f3314f64615d","Type":"ContainerDied","Data":"3c036acf871e4e7c1b650e37bc9622d291efdeb5ae96db5fa520af94f06423f8"}
Sep 19 01:18:58.160995 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:58.160404    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c036acf871e4e7c1b650e37bc9622d291efdeb5ae96db5fa520af94f06423f8"
Sep 19 01:18:58.162397 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:58.162363    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv"
Sep 19 01:18:58.162397 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:58.162380    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79dhlv" event={"ID":"b6f77707-1c65-4c81-be7f-0178cf865f20","Type":"ContainerDied","Data":"5ff6c75a49eae782ae0cd5998d6ff91069c3e1494fc3b501b55af9bc10cb5224"}
Sep 19 01:18:58.162636 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:58.162424    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5ff6c75a49eae782ae0cd5998d6ff91069c3e1494fc3b501b55af9bc10cb5224"
Sep 19 01:18:58.164526 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:58.164487    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n" event={"ID":"92df7e56-8584-4e44-a365-4fd75758e912","Type":"ContainerDied","Data":"b1235f8aea62faa30e937563c63acaf6d68a3b2b80d1b48699bc3b342e985aa8"}
Sep 19 01:18:58.164526 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:58.164521    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cz2t8n"
Sep 19 01:18:58.164659 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:18:58.164540    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b1235f8aea62faa30e937563c63acaf6d68a3b2b80d1b48699bc3b342e985aa8"
Sep 19 01:18:58.208185 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:18:58.208161    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92df7e56_8584_4e44_a365_4fd75758e912.slice\": RecentStats: unable to find data in memory cache]"
Sep 19 01:19:07.780417 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.780381    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"]
Sep 19 01:19:07.781211 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781187    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6f77707-1c65-4c81-be7f-0178cf865f20" containerName="util"
Sep 19 01:19:07.781329 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781214    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6f77707-1c65-4c81-be7f-0178cf865f20" containerName="util"
Sep 19 01:19:07.781329 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781239    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6f77707-1c65-4c81-be7f-0178cf865f20" containerName="extract"
Sep 19 01:19:07.781329 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781247    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6f77707-1c65-4c81-be7f-0178cf865f20" containerName="extract"
Sep 19 01:19:07.781329 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781268    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c4f0f08-8b4a-49fc-8a34-c80e89209690" containerName="extract"
Sep 19 01:19:07.781329 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781279    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c4f0f08-8b4a-49fc-8a34-c80e89209690" containerName="extract"
Sep 19 01:19:07.781329 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781307    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6f77707-1c65-4c81-be7f-0178cf865f20" containerName="pull"
Sep 19 01:19:07.781329 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781315    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6f77707-1c65-4c81-be7f-0178cf865f20" containerName="pull"
Sep 19 01:19:07.781329 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781330    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1029fd0f-2726-41b2-b2ba-f3314f64615d" containerName="pull"
Sep 19 01:19:07.781714 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781338    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1029fd0f-2726-41b2-b2ba-f3314f64615d" containerName="pull"
Sep 19 01:19:07.781714 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781349    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92df7e56-8584-4e44-a365-4fd75758e912" containerName="pull"
Sep 19 01:19:07.781714 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781356    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="92df7e56-8584-4e44-a365-4fd75758e912" containerName="pull"
Sep 19 01:19:07.781714 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781372    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1029fd0f-2726-41b2-b2ba-f3314f64615d" containerName="extract"
Sep 19 01:19:07.781714 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781379    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1029fd0f-2726-41b2-b2ba-f3314f64615d" containerName="extract"
Sep 19 01:19:07.781714 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781398    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92df7e56-8584-4e44-a365-4fd75758e912" containerName="util"
Sep 19 01:19:07.781714 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781406    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="92df7e56-8584-4e44-a365-4fd75758e912" containerName="util"
Sep 19 01:19:07.781714 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781421    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92df7e56-8584-4e44-a365-4fd75758e912" containerName="extract"
Sep 19 01:19:07.781714 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781429    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="92df7e56-8584-4e44-a365-4fd75758e912" containerName="extract"
Sep 19 01:19:07.781714 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781447    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c4f0f08-8b4a-49fc-8a34-c80e89209690" containerName="pull"
Sep 19 01:19:07.781714 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781454    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c4f0f08-8b4a-49fc-8a34-c80e89209690" containerName="pull"
Sep 19 01:19:07.781714 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781474    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1029fd0f-2726-41b2-b2ba-f3314f64615d" containerName="util"
Sep 19 01:19:07.781714 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781483    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1029fd0f-2726-41b2-b2ba-f3314f64615d" containerName="util"
Sep 19 01:19:07.781714 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781511    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c4f0f08-8b4a-49fc-8a34-c80e89209690" containerName="util"
Sep 19 01:19:07.781714 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781519    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c4f0f08-8b4a-49fc-8a34-c80e89209690" containerName="util"
Sep 19 01:19:07.781714 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781680    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="92df7e56-8584-4e44-a365-4fd75758e912" containerName="extract"
Sep 19 01:19:07.781714 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781692    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6f77707-1c65-4c81-be7f-0178cf865f20" containerName="extract"
Sep 19 01:19:07.781714 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781712    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1029fd0f-2726-41b2-b2ba-f3314f64615d" containerName="extract"
Sep 19 01:19:07.782484 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.781731    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c4f0f08-8b4a-49fc-8a34-c80e89209690" containerName="extract"
Sep 19 01:19:07.787685 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.787661    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"
Sep 19 01:19:07.792421 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.792398    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 19 01:19:07.792591 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.792570    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nfzf4\""
Sep 19 01:19:07.792645 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.792605    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 19 01:19:07.809482 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.809458    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"]
Sep 19 01:19:07.896829 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.896792    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkms6\" (UniqueName: \"kubernetes.io/projected/3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31-kube-api-access-jkms6\") pod \"kuadrant-operator-controller-manager-76f8f686bf-k9ggw\" (UID: \"3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"
Sep 19 01:19:07.897003 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.896857    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-k9ggw\" (UID: \"3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"
Sep 19 01:19:07.997714 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.997673    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-k9ggw\" (UID: \"3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"
Sep 19 01:19:07.997879 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.997773    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkms6\" (UniqueName: \"kubernetes.io/projected/3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31-kube-api-access-jkms6\") pod \"kuadrant-operator-controller-manager-76f8f686bf-k9ggw\" (UID: \"3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"
Sep 19 01:19:07.998055 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:07.998035    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-k9ggw\" (UID: \"3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"
Sep 19 01:19:08.007057 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.007026    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkms6\" (UniqueName: \"kubernetes.io/projected/3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31-kube-api-access-jkms6\") pod \"kuadrant-operator-controller-manager-76f8f686bf-k9ggw\" (UID: \"3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"
Sep 19 01:19:08.097846 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.097752    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"
Sep 19 01:19:08.300225 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.300189    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56c5d49fcb-2dtxc"]
Sep 19 01:19:08.305262 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.305240    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:08.319545 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.319522    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56c5d49fcb-2dtxc"]
Sep 19 01:19:08.386851 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.386831    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"]
Sep 19 01:19:08.389130 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:19:08.389106    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3833fc4c_2116_49f9_bb1d_ddc7f4c1bd31.slice/crio-2372e316cbf6810d7772448894364dfe4690a602204060d4fbb5ad12299e23ce WatchSource:0}: Error finding container 2372e316cbf6810d7772448894364dfe4690a602204060d4fbb5ad12299e23ce: Status 404 returned error can't find the container with id 2372e316cbf6810d7772448894364dfe4690a602204060d4fbb5ad12299e23ce
Sep 19 01:19:08.402455 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.402435    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1917c1fa-4d49-4356-b2ee-3f729d8866e4-console-oauth-config\") pod \"console-56c5d49fcb-2dtxc\" (UID: \"1917c1fa-4d49-4356-b2ee-3f729d8866e4\") " pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:08.402558 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.402466    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1917c1fa-4d49-4356-b2ee-3f729d8866e4-service-ca\") pod \"console-56c5d49fcb-2dtxc\" (UID: \"1917c1fa-4d49-4356-b2ee-3f729d8866e4\") " pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:08.402558 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.402530    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1917c1fa-4d49-4356-b2ee-3f729d8866e4-trusted-ca-bundle\") pod \"console-56c5d49fcb-2dtxc\" (UID: \"1917c1fa-4d49-4356-b2ee-3f729d8866e4\") " pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:08.402652 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.402637    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1917c1fa-4d49-4356-b2ee-3f729d8866e4-console-serving-cert\") pod \"console-56c5d49fcb-2dtxc\" (UID: \"1917c1fa-4d49-4356-b2ee-3f729d8866e4\") " pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:08.402735 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.402704    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thnvc\" (UniqueName: \"kubernetes.io/projected/1917c1fa-4d49-4356-b2ee-3f729d8866e4-kube-api-access-thnvc\") pod \"console-56c5d49fcb-2dtxc\" (UID: \"1917c1fa-4d49-4356-b2ee-3f729d8866e4\") " pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:08.402780 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.402735    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1917c1fa-4d49-4356-b2ee-3f729d8866e4-oauth-serving-cert\") pod \"console-56c5d49fcb-2dtxc\" (UID: \"1917c1fa-4d49-4356-b2ee-3f729d8866e4\") " pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:08.402817 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.402782    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1917c1fa-4d49-4356-b2ee-3f729d8866e4-console-config\") pod \"console-56c5d49fcb-2dtxc\" (UID: \"1917c1fa-4d49-4356-b2ee-3f729d8866e4\") " pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:08.503838 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.503812    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1917c1fa-4d49-4356-b2ee-3f729d8866e4-console-config\") pod \"console-56c5d49fcb-2dtxc\" (UID: \"1917c1fa-4d49-4356-b2ee-3f729d8866e4\") " pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:08.504001 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.503857    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1917c1fa-4d49-4356-b2ee-3f729d8866e4-console-oauth-config\") pod \"console-56c5d49fcb-2dtxc\" (UID: \"1917c1fa-4d49-4356-b2ee-3f729d8866e4\") " pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:08.504001 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.503878    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1917c1fa-4d49-4356-b2ee-3f729d8866e4-service-ca\") pod \"console-56c5d49fcb-2dtxc\" (UID: \"1917c1fa-4d49-4356-b2ee-3f729d8866e4\") " pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:08.504001 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.503911    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1917c1fa-4d49-4356-b2ee-3f729d8866e4-trusted-ca-bundle\") pod \"console-56c5d49fcb-2dtxc\" (UID: \"1917c1fa-4d49-4356-b2ee-3f729d8866e4\") " pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:08.504001 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.503955    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1917c1fa-4d49-4356-b2ee-3f729d8866e4-console-serving-cert\") pod \"console-56c5d49fcb-2dtxc\" (UID: \"1917c1fa-4d49-4356-b2ee-3f729d8866e4\") " pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:08.504246 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.504124    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thnvc\" (UniqueName: \"kubernetes.io/projected/1917c1fa-4d49-4356-b2ee-3f729d8866e4-kube-api-access-thnvc\") pod \"console-56c5d49fcb-2dtxc\" (UID: \"1917c1fa-4d49-4356-b2ee-3f729d8866e4\") " pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:08.504246 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.504178    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1917c1fa-4d49-4356-b2ee-3f729d8866e4-oauth-serving-cert\") pod \"console-56c5d49fcb-2dtxc\" (UID: \"1917c1fa-4d49-4356-b2ee-3f729d8866e4\") " pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:08.504752 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.504724    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1917c1fa-4d49-4356-b2ee-3f729d8866e4-console-config\") pod \"console-56c5d49fcb-2dtxc\" (UID: \"1917c1fa-4d49-4356-b2ee-3f729d8866e4\") " pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:08.504861 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.504733    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1917c1fa-4d49-4356-b2ee-3f729d8866e4-service-ca\") pod \"console-56c5d49fcb-2dtxc\" (UID: \"1917c1fa-4d49-4356-b2ee-3f729d8866e4\") " pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:08.504921 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.504902    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1917c1fa-4d49-4356-b2ee-3f729d8866e4-trusted-ca-bundle\") pod \"console-56c5d49fcb-2dtxc\" (UID: \"1917c1fa-4d49-4356-b2ee-3f729d8866e4\") " pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:08.504970 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.504912    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1917c1fa-4d49-4356-b2ee-3f729d8866e4-oauth-serving-cert\") pod \"console-56c5d49fcb-2dtxc\" (UID: \"1917c1fa-4d49-4356-b2ee-3f729d8866e4\") " pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:08.506578 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.506557    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1917c1fa-4d49-4356-b2ee-3f729d8866e4-console-oauth-config\") pod \"console-56c5d49fcb-2dtxc\" (UID: \"1917c1fa-4d49-4356-b2ee-3f729d8866e4\") " pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:08.506664 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.506557    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1917c1fa-4d49-4356-b2ee-3f729d8866e4-console-serving-cert\") pod \"console-56c5d49fcb-2dtxc\" (UID: \"1917c1fa-4d49-4356-b2ee-3f729d8866e4\") " pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:08.512100 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.512071    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thnvc\" (UniqueName: \"kubernetes.io/projected/1917c1fa-4d49-4356-b2ee-3f729d8866e4-kube-api-access-thnvc\") pod \"console-56c5d49fcb-2dtxc\" (UID: \"1917c1fa-4d49-4356-b2ee-3f729d8866e4\") " pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:08.615731 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.615694    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:08.925371 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:08.925348    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56c5d49fcb-2dtxc"]
Sep 19 01:19:08.927317 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:19:08.927292    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1917c1fa_4d49_4356_b2ee_3f729d8866e4.slice/crio-75e4c5d961839144abe5f430123430cb3510b64ca7ef7115e19487c44e1df24d WatchSource:0}: Error finding container 75e4c5d961839144abe5f430123430cb3510b64ca7ef7115e19487c44e1df24d: Status 404 returned error can't find the container with id 75e4c5d961839144abe5f430123430cb3510b64ca7ef7115e19487c44e1df24d
Sep 19 01:19:09.229453 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:09.229357    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56c5d49fcb-2dtxc" event={"ID":"1917c1fa-4d49-4356-b2ee-3f729d8866e4","Type":"ContainerStarted","Data":"1184fdf75445ee9daec3dee2fd4b4c1a214fece1a091c111ddcc47a07f1fdbee"}
Sep 19 01:19:09.229453 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:09.229415    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56c5d49fcb-2dtxc" event={"ID":"1917c1fa-4d49-4356-b2ee-3f729d8866e4","Type":"ContainerStarted","Data":"75e4c5d961839144abe5f430123430cb3510b64ca7ef7115e19487c44e1df24d"}
Sep 19 01:19:09.230787 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:09.230760    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw" event={"ID":"3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31","Type":"ContainerStarted","Data":"2372e316cbf6810d7772448894364dfe4690a602204060d4fbb5ad12299e23ce"}
Sep 19 01:19:09.260559 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:09.260523    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56c5d49fcb-2dtxc" podStartSLOduration=1.260507811 podStartE2EDuration="1.260507811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:19:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:19:09.258964843 +0000 UTC m=+404.165826892" watchObservedRunningTime="2026-09-19 01:19:09.260507811 +0000 UTC m=+404.167369849"
Sep 19 01:19:09.716405 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:09.716369    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7"]
Sep 19 01:19:09.720399 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:09.720375    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7"
Sep 19 01:19:09.723730 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:09.723698    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-bqsz2\""
Sep 19 01:19:09.740829 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:09.740783    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7"]
Sep 19 01:19:09.821691 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:09.821652    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w66th\" (UniqueName: \"kubernetes.io/projected/82bf629f-aa66-4ba6-bf3c-eed126e47ef6-kube-api-access-w66th\") pod \"limitador-operator-controller-manager-74495c69f8-pxpp7\" (UID: \"82bf629f-aa66-4ba6-bf3c-eed126e47ef6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7"
Sep 19 01:19:09.922863 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:09.922825    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w66th\" (UniqueName: \"kubernetes.io/projected/82bf629f-aa66-4ba6-bf3c-eed126e47ef6-kube-api-access-w66th\") pod \"limitador-operator-controller-manager-74495c69f8-pxpp7\" (UID: \"82bf629f-aa66-4ba6-bf3c-eed126e47ef6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7"
Sep 19 01:19:09.940045 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:09.940015    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w66th\" (UniqueName: \"kubernetes.io/projected/82bf629f-aa66-4ba6-bf3c-eed126e47ef6-kube-api-access-w66th\") pod \"limitador-operator-controller-manager-74495c69f8-pxpp7\" (UID: \"82bf629f-aa66-4ba6-bf3c-eed126e47ef6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7"
Sep 19 01:19:10.036390 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:10.036351    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7"
Sep 19 01:19:10.348744 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:10.348713    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7"]
Sep 19 01:19:10.350700 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:19:10.350662    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82bf629f_aa66_4ba6_bf3c_eed126e47ef6.slice/crio-625374da7f92f54e707ae1cd852a5692e0abc7692091cf1e5bad31f00334515e WatchSource:0}: Error finding container 625374da7f92f54e707ae1cd852a5692e0abc7692091cf1e5bad31f00334515e: Status 404 returned error can't find the container with id 625374da7f92f54e707ae1cd852a5692e0abc7692091cf1e5bad31f00334515e
Sep 19 01:19:11.250470 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:11.250421    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7" event={"ID":"82bf629f-aa66-4ba6-bf3c-eed126e47ef6","Type":"ContainerStarted","Data":"625374da7f92f54e707ae1cd852a5692e0abc7692091cf1e5bad31f00334515e"}
Sep 19 01:19:11.877739 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:11.877705    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zgxhm"]
Sep 19 01:19:11.882017 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:11.881994    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zgxhm"
Sep 19 01:19:11.884390 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:11.884364    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 19 01:19:11.884541 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:11.884486    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-tbd9b\""
Sep 19 01:19:11.899916 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:11.899862    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zgxhm"]
Sep 19 01:19:11.943381 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:11.943329    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktd8t\" (UniqueName: \"kubernetes.io/projected/5ca3e984-22f7-4867-8d0b-0c61c8d9c6cf-kube-api-access-ktd8t\") pod \"dns-operator-controller-manager-7b84cd7c9c-zgxhm\" (UID: \"5ca3e984-22f7-4867-8d0b-0c61c8d9c6cf\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zgxhm"
Sep 19 01:19:12.044879 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:12.044840    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktd8t\" (UniqueName: \"kubernetes.io/projected/5ca3e984-22f7-4867-8d0b-0c61c8d9c6cf-kube-api-access-ktd8t\") pod \"dns-operator-controller-manager-7b84cd7c9c-zgxhm\" (UID: \"5ca3e984-22f7-4867-8d0b-0c61c8d9c6cf\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zgxhm"
Sep 19 01:19:12.054642 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:12.054612    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktd8t\" (UniqueName: \"kubernetes.io/projected/5ca3e984-22f7-4867-8d0b-0c61c8d9c6cf-kube-api-access-ktd8t\") pod \"dns-operator-controller-manager-7b84cd7c9c-zgxhm\" (UID: \"5ca3e984-22f7-4867-8d0b-0c61c8d9c6cf\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zgxhm"
Sep 19 01:19:12.196232 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:12.196161    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zgxhm"
Sep 19 01:19:13.783085 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:13.783060    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zgxhm"]
Sep 19 01:19:13.784735 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:19:13.784713    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ca3e984_22f7_4867_8d0b_0c61c8d9c6cf.slice/crio-862865345bd24b5a068c088b1ab0f87292d1a67a059dce1e2bf1bed63c5eed25 WatchSource:0}: Error finding container 862865345bd24b5a068c088b1ab0f87292d1a67a059dce1e2bf1bed63c5eed25: Status 404 returned error can't find the container with id 862865345bd24b5a068c088b1ab0f87292d1a67a059dce1e2bf1bed63c5eed25
Sep 19 01:19:14.268779 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:14.268744    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zgxhm" event={"ID":"5ca3e984-22f7-4867-8d0b-0c61c8d9c6cf","Type":"ContainerStarted","Data":"862865345bd24b5a068c088b1ab0f87292d1a67a059dce1e2bf1bed63c5eed25"}
Sep 19 01:19:14.270476 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:14.270450    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7" event={"ID":"82bf629f-aa66-4ba6-bf3c-eed126e47ef6","Type":"ContainerStarted","Data":"c25f7d3991d2655dfba3c83c63c0f79ff0fae70205600e7ca02e84286edb2486"}
Sep 19 01:19:14.270633 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:14.270553    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7"
Sep 19 01:19:14.272200 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:14.272175    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw" event={"ID":"3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31","Type":"ContainerStarted","Data":"82bccfd38d401657bf9b36eb6231e25a96791675a90be488823a0c447d754690"}
Sep 19 01:19:14.272312 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:14.272299    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"
Sep 19 01:19:14.293574 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:14.293533    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7" podStartSLOduration=2.139093944 podStartE2EDuration="5.293517944s" totalImagesPullingTime="3.154424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:19:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:19:14.291550819 +0000 UTC m=+409.198412867" watchObservedRunningTime="2026-09-19 01:19:14.293517944 +0000 UTC m=+409.200379990"
Sep 19 01:19:14.324667 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:14.324617    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw" podStartSLOduration=2.190199714 podStartE2EDuration="7.324599126s" totalImagesPullingTime="5.134399412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:19:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:19:14.322857497 +0000 UTC m=+409.229719568" watchObservedRunningTime="2026-09-19 01:19:14.324599126 +0000 UTC m=+409.231461175"
Sep 19 01:19:14.729050 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:14.728949    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-rmhw5"]
Sep 19 01:19:14.733587 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:14.733556    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-rmhw5"
Sep 19 01:19:14.739124 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:14.739076    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-7m8n6\""
Sep 19 01:19:14.755126 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:14.755099    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-rmhw5"]
Sep 19 01:19:14.772017 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:14.771132    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z6q4\" (UniqueName: \"kubernetes.io/projected/4370cd9a-5066-4f21-bbc9-620ecfdcd6ff-kube-api-access-4z6q4\") pod \"authorino-operator-76b478bd78-rmhw5\" (UID: \"4370cd9a-5066-4f21-bbc9-620ecfdcd6ff\") " pod="kuadrant-system/authorino-operator-76b478bd78-rmhw5"
Sep 19 01:19:14.871850 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:14.871819    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z6q4\" (UniqueName: \"kubernetes.io/projected/4370cd9a-5066-4f21-bbc9-620ecfdcd6ff-kube-api-access-4z6q4\") pod \"authorino-operator-76b478bd78-rmhw5\" (UID: \"4370cd9a-5066-4f21-bbc9-620ecfdcd6ff\") " pod="kuadrant-system/authorino-operator-76b478bd78-rmhw5"
Sep 19 01:19:14.883399 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:14.883342    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z6q4\" (UniqueName: \"kubernetes.io/projected/4370cd9a-5066-4f21-bbc9-620ecfdcd6ff-kube-api-access-4z6q4\") pod \"authorino-operator-76b478bd78-rmhw5\" (UID: \"4370cd9a-5066-4f21-bbc9-620ecfdcd6ff\") " pod="kuadrant-system/authorino-operator-76b478bd78-rmhw5"
Sep 19 01:19:15.048322 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:15.048289    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-rmhw5"
Sep 19 01:19:15.687042 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:15.687018    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-rmhw5"]
Sep 19 01:19:15.688966 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:19:15.688941    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4370cd9a_5066_4f21_bbc9_620ecfdcd6ff.slice/crio-46241e847b937ccc7b846e34d63731e1c10e650b41c54bf095f6f46c9d170e62 WatchSource:0}: Error finding container 46241e847b937ccc7b846e34d63731e1c10e650b41c54bf095f6f46c9d170e62: Status 404 returned error can't find the container with id 46241e847b937ccc7b846e34d63731e1c10e650b41c54bf095f6f46c9d170e62
Sep 19 01:19:16.292460 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:16.292418    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zgxhm" event={"ID":"5ca3e984-22f7-4867-8d0b-0c61c8d9c6cf","Type":"ContainerStarted","Data":"3b50a0765e0d31c8d5f95ffadc72a969ea6133c316b2483761cd2f0175258abb"}
Sep 19 01:19:16.292940 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:16.292541    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zgxhm"
Sep 19 01:19:16.293876 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:16.293855    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-rmhw5" event={"ID":"4370cd9a-5066-4f21-bbc9-620ecfdcd6ff","Type":"ContainerStarted","Data":"46241e847b937ccc7b846e34d63731e1c10e650b41c54bf095f6f46c9d170e62"}
Sep 19 01:19:16.310576 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:16.310532    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zgxhm" podStartSLOduration=3.643141775 podStartE2EDuration="5.310517691s" totalImagesPullingTime="1.667375916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:19:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:19:16.308854684 +0000 UTC m=+411.215716733" watchObservedRunningTime="2026-09-19 01:19:16.310517691 +0000 UTC m=+411.217379735"
Sep 19 01:19:18.307643 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:18.307605    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-rmhw5" event={"ID":"4370cd9a-5066-4f21-bbc9-620ecfdcd6ff","Type":"ContainerStarted","Data":"f4c85516ac1f1b5703976cb9cf07455b690f2a882233fdfa1e0bce274790b61b"}
Sep 19 01:19:18.308105 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:18.307741    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-rmhw5"
Sep 19 01:19:18.330610 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:18.330558    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-rmhw5" podStartSLOduration=2.353387866 podStartE2EDuration="4.330542483s" totalImagesPullingTime="1.977154617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:19:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:19:18.32722455 +0000 UTC m=+413.234086599" watchObservedRunningTime="2026-09-19 01:19:18.330542483 +0000 UTC m=+413.237404527"
Sep 19 01:19:18.616589 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:18.616482    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:18.616589 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:18.616554    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:18.621306 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:18.621283    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:19.316846 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:19.316816    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56c5d49fcb-2dtxc"
Sep 19 01:19:19.409108 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:19.409079    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bfb8b856-s8hkr"]
Sep 19 01:19:25.281690 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:25.281647    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"
Sep 19 01:19:25.282138 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:25.281896    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7"
Sep 19 01:19:27.279905 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.279874    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-rhbdj"]
Sep 19 01:19:27.285097 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.285076    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-rhbdj"
Sep 19 01:19:27.299018 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.298997    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-rhbdj"]
Sep 19 01:19:27.302584 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.302561    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zgxhm"
Sep 19 01:19:27.390676 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.390649    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74aaa91b-a7ce-4e2a-ac70-863921e21ed3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-rhbdj\" (UID: \"74aaa91b-a7ce-4e2a-ac70-863921e21ed3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-rhbdj"
Sep 19 01:19:27.390814 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.390702    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvgnb\" (UniqueName: \"kubernetes.io/projected/74aaa91b-a7ce-4e2a-ac70-863921e21ed3-kube-api-access-rvgnb\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-rhbdj\" (UID: \"74aaa91b-a7ce-4e2a-ac70-863921e21ed3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-rhbdj"
Sep 19 01:19:27.459463 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.459435    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"]
Sep 19 01:19:27.459685 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.459662    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw" podUID="3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31" containerName="manager" containerID="cri-o://82bccfd38d401657bf9b36eb6231e25a96791675a90be488823a0c447d754690" gracePeriod=2
Sep 19 01:19:27.470887 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.470862    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"]
Sep 19 01:19:27.490073 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.490001    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-rhbdj"]
Sep 19 01:19:27.490384 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:27.490355    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-rvgnb], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-rhbdj" podUID="74aaa91b-a7ce-4e2a-ac70-863921e21ed3"
Sep 19 01:19:27.491999 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.491963    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74aaa91b-a7ce-4e2a-ac70-863921e21ed3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-rhbdj\" (UID: \"74aaa91b-a7ce-4e2a-ac70-863921e21ed3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-rhbdj"
Sep 19 01:19:27.492154 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.492026    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvgnb\" (UniqueName: \"kubernetes.io/projected/74aaa91b-a7ce-4e2a-ac70-863921e21ed3-kube-api-access-rvgnb\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-rhbdj\" (UID: \"74aaa91b-a7ce-4e2a-ac70-863921e21ed3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-rhbdj"
Sep 19 01:19:27.492459 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.492436    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74aaa91b-a7ce-4e2a-ac70-863921e21ed3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-rhbdj\" (UID: \"74aaa91b-a7ce-4e2a-ac70-863921e21ed3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-rhbdj"
Sep 19 01:19:27.495157 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:27.495134    2580 projected.go:196] Error preparing data for projected volume kube-api-access-rvgnb for pod kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-rhbdj: failed to fetch token: serviceaccounts "kuadrant-operator-controller-manager" is forbidden: User "system:node:ip-10-0-131-76.ec2.internal" cannot create resource "serviceaccounts/token" in API group "" in the namespace "kuadrant-system": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object
Sep 19 01:19:27.495275 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:27.495199    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74aaa91b-a7ce-4e2a-ac70-863921e21ed3-kube-api-access-rvgnb podName:74aaa91b-a7ce-4e2a-ac70-863921e21ed3 nodeName:}" failed. No retries permitted until 2026-09-19 01:19:27.995177615 +0000 UTC m=+422.902039652 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rvgnb" (UniqueName: "kubernetes.io/projected/74aaa91b-a7ce-4e2a-ac70-863921e21ed3-kube-api-access-rvgnb") pod "kuadrant-operator-controller-manager-7978cbdfbc-rhbdj" (UID: "74aaa91b-a7ce-4e2a-ac70-863921e21ed3") : failed to fetch token: serviceaccounts "kuadrant-operator-controller-manager" is forbidden: User "system:node:ip-10-0-131-76.ec2.internal" cannot create resource "serviceaccounts/token" in API group "" in the namespace "kuadrant-system": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object
Sep 19 01:19:27.495657 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.495637    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-rhbdj"]
Sep 19 01:19:27.500450 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.500430    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-sd45d"]
Sep 19 01:19:27.501024 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.501007    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31" containerName="manager"
Sep 19 01:19:27.501024 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.501025    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31" containerName="manager"
Sep 19 01:19:27.501160 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.501143    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31" containerName="manager"
Sep 19 01:19:27.504620 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.504604    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-sd45d"
Sep 19 01:19:27.517002 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.516966    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7"]
Sep 19 01:19:27.517250 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.517214    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7" podUID="82bf629f-aa66-4ba6-bf3c-eed126e47ef6" containerName="manager" containerID="cri-o://c25f7d3991d2655dfba3c83c63c0f79ff0fae70205600e7ca02e84286edb2486" gracePeriod=2
Sep 19 01:19:27.520865 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.520842    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-sd45d"]
Sep 19 01:19:27.538260 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.538199    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7"]
Sep 19 01:19:27.553657 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:27.553628    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-k9ggw\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"
Sep 19 01:19:27.553809 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.553785    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zzrps"]
Sep 19 01:19:27.554411 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.554394    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82bf629f-aa66-4ba6-bf3c-eed126e47ef6" containerName="manager"
Sep 19 01:19:27.554486 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.554414    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="82bf629f-aa66-4ba6-bf3c-eed126e47ef6" containerName="manager"
Sep 19 01:19:27.554563 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.554523    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="82bf629f-aa66-4ba6-bf3c-eed126e47ef6" containerName="manager"
Sep 19 01:19:27.555923 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:27.555878    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-pxpp7\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="82bf629f-aa66-4ba6-bf3c-eed126e47ef6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7"
Sep 19 01:19:27.558017 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:27.557991    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-k9ggw\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"
Sep 19 01:19:27.558100 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.558089    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zzrps"
Sep 19 01:19:27.564287 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:27.564259    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-pxpp7\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="82bf629f-aa66-4ba6-bf3c-eed126e47ef6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7"
Sep 19 01:19:27.566062 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:27.566037    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-k9ggw\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"
Sep 19 01:19:27.567563 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.567541    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-svhp6"]
Sep 19 01:19:27.570987 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.570966    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-svhp6"
Sep 19 01:19:27.575966 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.575943    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zzrps"]
Sep 19 01:19:27.593414 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.593380    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0065e9b0-1ee9-4405-b9e9-dcde83ffa74b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-sd45d\" (UID: \"0065e9b0-1ee9-4405-b9e9-dcde83ffa74b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-sd45d"
Sep 19 01:19:27.593586 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.593565    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw6gs\" (UniqueName: \"kubernetes.io/projected/0065e9b0-1ee9-4405-b9e9-dcde83ffa74b-kube-api-access-lw6gs\") pod \"kuadrant-operator-controller-manager-76f8f686bf-sd45d\" (UID: \"0065e9b0-1ee9-4405-b9e9-dcde83ffa74b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-sd45d"
Sep 19 01:19:27.594323 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.594298    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-svhp6"]
Sep 19 01:19:27.601385 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:27.601357    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-pxpp7\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="82bf629f-aa66-4ba6-bf3c-eed126e47ef6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7"
Sep 19 01:19:27.603441 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:27.603415    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-k9ggw\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"
Sep 19 01:19:27.696034 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.695184    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zzrps\" (UID: \"5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zzrps"
Sep 19 01:19:27.696034 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.695269    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw6gs\" (UniqueName: \"kubernetes.io/projected/0065e9b0-1ee9-4405-b9e9-dcde83ffa74b-kube-api-access-lw6gs\") pod \"kuadrant-operator-controller-manager-76f8f686bf-sd45d\" (UID: \"0065e9b0-1ee9-4405-b9e9-dcde83ffa74b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-sd45d"
Sep 19 01:19:27.696034 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.695343    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vlqw\" (UniqueName: \"kubernetes.io/projected/9224c100-d688-4875-977f-95680c0778ac-kube-api-access-7vlqw\") pod \"limitador-operator-controller-manager-74495c69f8-svhp6\" (UID: \"9224c100-d688-4875-977f-95680c0778ac\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-svhp6"
Sep 19 01:19:27.696034 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.695420    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq2tj\" (UniqueName: \"kubernetes.io/projected/5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb-kube-api-access-mq2tj\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zzrps\" (UID: \"5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zzrps"
Sep 19 01:19:27.696034 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.695449    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0065e9b0-1ee9-4405-b9e9-dcde83ffa74b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-sd45d\" (UID: \"0065e9b0-1ee9-4405-b9e9-dcde83ffa74b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-sd45d"
Sep 19 01:19:27.696034 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.695897    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0065e9b0-1ee9-4405-b9e9-dcde83ffa74b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-sd45d\" (UID: \"0065e9b0-1ee9-4405-b9e9-dcde83ffa74b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-sd45d"
Sep 19 01:19:27.710582 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.710546    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw6gs\" (UniqueName: \"kubernetes.io/projected/0065e9b0-1ee9-4405-b9e9-dcde83ffa74b-kube-api-access-lw6gs\") pod \"kuadrant-operator-controller-manager-76f8f686bf-sd45d\" (UID: \"0065e9b0-1ee9-4405-b9e9-dcde83ffa74b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-sd45d"
Sep 19 01:19:27.767669 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.767649    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"
Sep 19 01:19:27.769744 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:27.769713    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-pxpp7\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="82bf629f-aa66-4ba6-bf3c-eed126e47ef6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7"
Sep 19 01:19:27.770402 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.770388    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7"
Sep 19 01:19:27.771461 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:27.771440    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-k9ggw\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"
Sep 19 01:19:27.773817 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:27.773796    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-pxpp7\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="82bf629f-aa66-4ba6-bf3c-eed126e47ef6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7"
Sep 19 01:19:27.775655 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:27.775638    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-k9ggw\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"
Sep 19 01:19:27.796115 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.796050    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zzrps\" (UID: \"5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zzrps"
Sep 19 01:19:27.796195 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.796155    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vlqw\" (UniqueName: \"kubernetes.io/projected/9224c100-d688-4875-977f-95680c0778ac-kube-api-access-7vlqw\") pod \"limitador-operator-controller-manager-74495c69f8-svhp6\" (UID: \"9224c100-d688-4875-977f-95680c0778ac\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-svhp6"
Sep 19 01:19:27.796244 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.796205    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq2tj\" (UniqueName: \"kubernetes.io/projected/5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb-kube-api-access-mq2tj\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zzrps\" (UID: \"5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zzrps"
Sep 19 01:19:27.796382 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.796367    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zzrps\" (UID: \"5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zzrps"
Sep 19 01:19:27.811188 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.811159    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vlqw\" (UniqueName: \"kubernetes.io/projected/9224c100-d688-4875-977f-95680c0778ac-kube-api-access-7vlqw\") pod \"limitador-operator-controller-manager-74495c69f8-svhp6\" (UID: \"9224c100-d688-4875-977f-95680c0778ac\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-svhp6"
Sep 19 01:19:27.811273 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.811196    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq2tj\" (UniqueName: \"kubernetes.io/projected/5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb-kube-api-access-mq2tj\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zzrps\" (UID: \"5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zzrps"
Sep 19 01:19:27.889282 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.889253    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-sd45d"
Sep 19 01:19:27.897418 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.897396    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkms6\" (UniqueName: \"kubernetes.io/projected/3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31-kube-api-access-jkms6\") pod \"3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31\" (UID: \"3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31\") "
Sep 19 01:19:27.897543 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.897466    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31-extensions-socket-volume\") pod \"3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31\" (UID: \"3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31\") "
Sep 19 01:19:27.897608 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.897581    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w66th\" (UniqueName: \"kubernetes.io/projected/82bf629f-aa66-4ba6-bf3c-eed126e47ef6-kube-api-access-w66th\") pod \"82bf629f-aa66-4ba6-bf3c-eed126e47ef6\" (UID: \"82bf629f-aa66-4ba6-bf3c-eed126e47ef6\") "
Sep 19 01:19:27.897854 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.897833    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31" (UID: "3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:19:27.899668 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.899645    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31-kube-api-access-jkms6" (OuterVolumeSpecName: "kube-api-access-jkms6") pod "3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31" (UID: "3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31"). InnerVolumeSpecName "kube-api-access-jkms6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:19:27.899763 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.899743    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82bf629f-aa66-4ba6-bf3c-eed126e47ef6-kube-api-access-w66th" (OuterVolumeSpecName: "kube-api-access-w66th") pod "82bf629f-aa66-4ba6-bf3c-eed126e47ef6" (UID: "82bf629f-aa66-4ba6-bf3c-eed126e47ef6"). InnerVolumeSpecName "kube-api-access-w66th". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:19:27.903189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.903169    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zzrps"
Sep 19 01:19:27.909168 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.909152    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-svhp6"
Sep 19 01:19:27.998970 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.998937    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvgnb\" (UniqueName: \"kubernetes.io/projected/74aaa91b-a7ce-4e2a-ac70-863921e21ed3-kube-api-access-rvgnb\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-rhbdj\" (UID: \"74aaa91b-a7ce-4e2a-ac70-863921e21ed3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-rhbdj"
Sep 19 01:19:27.999072 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.999015    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkms6\" (UniqueName: \"kubernetes.io/projected/3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31-kube-api-access-jkms6\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:19:27.999072 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.999033    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31-extensions-socket-volume\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:19:27.999072 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:27.999047    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w66th\" (UniqueName: \"kubernetes.io/projected/82bf629f-aa66-4ba6-bf3c-eed126e47ef6-kube-api-access-w66th\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:19:28.006211 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:28.006191    2580 projected.go:196] Error preparing data for projected volume kube-api-access-rvgnb for pod kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-rhbdj: failed to fetch token: pod "kuadrant-operator-controller-manager-7978cbdfbc-rhbdj" not found
Sep 19 01:19:28.006296 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:28.006264    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74aaa91b-a7ce-4e2a-ac70-863921e21ed3-kube-api-access-rvgnb podName:74aaa91b-a7ce-4e2a-ac70-863921e21ed3 nodeName:}" failed. No retries permitted until 2026-09-19 01:19:29.006244611 +0000 UTC m=+423.913106645 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rvgnb" (UniqueName: "kubernetes.io/projected/74aaa91b-a7ce-4e2a-ac70-863921e21ed3-kube-api-access-rvgnb") pod "kuadrant-operator-controller-manager-7978cbdfbc-rhbdj" (UID: "74aaa91b-a7ce-4e2a-ac70-863921e21ed3") : failed to fetch token: pod "kuadrant-operator-controller-manager-7978cbdfbc-rhbdj" not found
Sep 19 01:19:28.182079 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.182054    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-sd45d"]
Sep 19 01:19:28.183746 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:19:28.183716    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0065e9b0_1ee9_4405_b9e9_dcde83ffa74b.slice/crio-24309ce2861c6946c4abffd8c45d756b35fc2bd70982e0a9e07eec488ea97637 WatchSource:0}: Error finding container 24309ce2861c6946c4abffd8c45d756b35fc2bd70982e0a9e07eec488ea97637: Status 404 returned error can't find the container with id 24309ce2861c6946c4abffd8c45d756b35fc2bd70982e0a9e07eec488ea97637
Sep 19 01:19:28.219294 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.219263    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-svhp6"]
Sep 19 01:19:28.221318 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:19:28.221283    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9224c100_d688_4875_977f_95680c0778ac.slice/crio-370a3d749a995164c521b52c6dfaade03c84964cdbc660ad3d11f7e8bf21edf4 WatchSource:0}: Error finding container 370a3d749a995164c521b52c6dfaade03c84964cdbc660ad3d11f7e8bf21edf4: Status 404 returned error can't find the container with id 370a3d749a995164c521b52c6dfaade03c84964cdbc660ad3d11f7e8bf21edf4
Sep 19 01:19:28.221505 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.221469    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zzrps"]
Sep 19 01:19:28.222253 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:19:28.222234    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c9a9323_c3dd_45cc_b7e0_cb4b7d8dfdeb.slice/crio-6d8c6db307bc7700ec153fb585a8d7775f9cfeb261749fd81aa12818edbaf110 WatchSource:0}: Error finding container 6d8c6db307bc7700ec153fb585a8d7775f9cfeb261749fd81aa12818edbaf110: Status 404 returned error can't find the container with id 6d8c6db307bc7700ec153fb585a8d7775f9cfeb261749fd81aa12818edbaf110
Sep 19 01:19:28.366754 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.366721    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-svhp6" event={"ID":"9224c100-d688-4875-977f-95680c0778ac","Type":"ContainerStarted","Data":"d57acd2de4a7fbb7aa03f75c957816fd4d08a4b634d06276fb6afc81145899ac"}
Sep 19 01:19:28.367187 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.366799    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-svhp6"
Sep 19 01:19:28.367187 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.366825    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-svhp6" event={"ID":"9224c100-d688-4875-977f-95680c0778ac","Type":"ContainerStarted","Data":"370a3d749a995164c521b52c6dfaade03c84964cdbc660ad3d11f7e8bf21edf4"}
Sep 19 01:19:28.368648 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.368617    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zzrps" event={"ID":"5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb","Type":"ContainerStarted","Data":"8a6c4180c14ab21e029612ac58a40988a9e1cac429fbb0458bd13ea04148e250"}
Sep 19 01:19:28.368814 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.368674    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zzrps"
Sep 19 01:19:28.368814 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.368697    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zzrps" event={"ID":"5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb","Type":"ContainerStarted","Data":"6d8c6db307bc7700ec153fb585a8d7775f9cfeb261749fd81aa12818edbaf110"}
Sep 19 01:19:28.368973 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:28.368949    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-pxpp7\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="82bf629f-aa66-4ba6-bf3c-eed126e47ef6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7"
Sep 19 01:19:28.370054 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.370029    2580 generic.go:356] "Generic (PLEG): container finished" podID="82bf629f-aa66-4ba6-bf3c-eed126e47ef6" containerID="c25f7d3991d2655dfba3c83c63c0f79ff0fae70205600e7ca02e84286edb2486" exitCode=0
Sep 19 01:19:28.370143 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.370066    2580 scope.go:117] "RemoveContainer" containerID="c25f7d3991d2655dfba3c83c63c0f79ff0fae70205600e7ca02e84286edb2486"
Sep 19 01:19:28.370143 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.370084    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7"
Sep 19 01:19:28.370682 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:28.370654    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-k9ggw\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"
Sep 19 01:19:28.371753 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.371727    2580 generic.go:356] "Generic (PLEG): container finished" podID="3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31" containerID="82bccfd38d401657bf9b36eb6231e25a96791675a90be488823a0c447d754690" exitCode=0
Sep 19 01:19:28.371853 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.371777    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"
Sep 19 01:19:28.373897 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.373865    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-sd45d" event={"ID":"0065e9b0-1ee9-4405-b9e9-dcde83ffa74b","Type":"ContainerStarted","Data":"b425132b0d795f9e011fde1552ba4d888accd80d710c2d9087063cc8264ad703"}
Sep 19 01:19:28.374054 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.373899    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-rhbdj"
Sep 19 01:19:28.374054 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.373921    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-sd45d" event={"ID":"0065e9b0-1ee9-4405-b9e9-dcde83ffa74b","Type":"ContainerStarted","Data":"24309ce2861c6946c4abffd8c45d756b35fc2bd70982e0a9e07eec488ea97637"}
Sep 19 01:19:28.374054 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.373969    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-sd45d"
Sep 19 01:19:28.381568 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.381550    2580 scope.go:117] "RemoveContainer" containerID="c25f7d3991d2655dfba3c83c63c0f79ff0fae70205600e7ca02e84286edb2486"
Sep 19 01:19:28.381827 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:28.381811    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c25f7d3991d2655dfba3c83c63c0f79ff0fae70205600e7ca02e84286edb2486\": container with ID starting with c25f7d3991d2655dfba3c83c63c0f79ff0fae70205600e7ca02e84286edb2486 not found: ID does not exist" containerID="c25f7d3991d2655dfba3c83c63c0f79ff0fae70205600e7ca02e84286edb2486"
Sep 19 01:19:28.381877 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.381836    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c25f7d3991d2655dfba3c83c63c0f79ff0fae70205600e7ca02e84286edb2486"} err="failed to get container status \"c25f7d3991d2655dfba3c83c63c0f79ff0fae70205600e7ca02e84286edb2486\": rpc error: code = NotFound desc = could not find container \"c25f7d3991d2655dfba3c83c63c0f79ff0fae70205600e7ca02e84286edb2486\": container with ID starting with c25f7d3991d2655dfba3c83c63c0f79ff0fae70205600e7ca02e84286edb2486 not found: ID does not exist"
Sep 19 01:19:28.381877 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.381851    2580 scope.go:117] "RemoveContainer" containerID="82bccfd38d401657bf9b36eb6231e25a96791675a90be488823a0c447d754690"
Sep 19 01:19:28.382593 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.382576    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-rhbdj"
Sep 19 01:19:28.393102 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.393082    2580 scope.go:117] "RemoveContainer" containerID="82bccfd38d401657bf9b36eb6231e25a96791675a90be488823a0c447d754690"
Sep 19 01:19:28.393693 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:28.393672    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82bccfd38d401657bf9b36eb6231e25a96791675a90be488823a0c447d754690\": container with ID starting with 82bccfd38d401657bf9b36eb6231e25a96791675a90be488823a0c447d754690 not found: ID does not exist" containerID="82bccfd38d401657bf9b36eb6231e25a96791675a90be488823a0c447d754690"
Sep 19 01:19:28.393772 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.393697    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82bccfd38d401657bf9b36eb6231e25a96791675a90be488823a0c447d754690"} err="failed to get container status \"82bccfd38d401657bf9b36eb6231e25a96791675a90be488823a0c447d754690\": rpc error: code = NotFound desc = could not find container \"82bccfd38d401657bf9b36eb6231e25a96791675a90be488823a0c447d754690\": container with ID starting with 82bccfd38d401657bf9b36eb6231e25a96791675a90be488823a0c447d754690 not found: ID does not exist"
Sep 19 01:19:28.394955 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:28.394932    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-rhbdj\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="74aaa91b-a7ce-4e2a-ac70-863921e21ed3" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-rhbdj"
Sep 19 01:19:28.395406 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.395373    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-svhp6" podStartSLOduration=1.395360613 podStartE2EDuration="1.395360613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:19:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:19:28.393300517 +0000 UTC m=+423.300162571" watchObservedRunningTime="2026-09-19 01:19:28.395360613 +0000 UTC m=+423.302222660"
Sep 19 01:19:28.421165 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:28.421139    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-pxpp7\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="82bf629f-aa66-4ba6-bf3c-eed126e47ef6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pxpp7"
Sep 19 01:19:28.421319 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.421289    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-sd45d" podStartSLOduration=1.42127933 podStartE2EDuration="1.42127933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:19:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:19:28.419422539 +0000 UTC m=+423.326284600" watchObservedRunningTime="2026-09-19 01:19:28.42127933 +0000 UTC m=+423.328141378"
Sep 19 01:19:28.423060 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:28.423040    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-k9ggw\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k9ggw"
Sep 19 01:19:28.451412 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.451313    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zzrps" podStartSLOduration=1.451296375 podStartE2EDuration="1.451296375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:19:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:19:28.449288011 +0000 UTC m=+423.356150060" watchObservedRunningTime="2026-09-19 01:19:28.451296375 +0000 UTC m=+423.358158423"
Sep 19 01:19:28.502465 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.502436    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74aaa91b-a7ce-4e2a-ac70-863921e21ed3-extensions-socket-volume\") pod \"74aaa91b-a7ce-4e2a-ac70-863921e21ed3\" (UID: \"74aaa91b-a7ce-4e2a-ac70-863921e21ed3\") "
Sep 19 01:19:28.502690 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.502667    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74aaa91b-a7ce-4e2a-ac70-863921e21ed3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "74aaa91b-a7ce-4e2a-ac70-863921e21ed3" (UID: "74aaa91b-a7ce-4e2a-ac70-863921e21ed3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:19:28.502791 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.502776    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvgnb\" (UniqueName: \"kubernetes.io/projected/74aaa91b-a7ce-4e2a-ac70-863921e21ed3-kube-api-access-rvgnb\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:19:28.502834 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:28.502793    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74aaa91b-a7ce-4e2a-ac70-863921e21ed3-extensions-socket-volume\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:19:29.064112 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:29.064065    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31" path="/var/lib/kubelet/pods/3833fc4c-2116-49f9-bb1d-ddc7f4c1bd31/volumes"
Sep 19 01:19:29.064432 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:29.064419    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74aaa91b-a7ce-4e2a-ac70-863921e21ed3" path="/var/lib/kubelet/pods/74aaa91b-a7ce-4e2a-ac70-863921e21ed3/volumes"
Sep 19 01:19:29.064636 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:29.064625    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82bf629f-aa66-4ba6-bf3c-eed126e47ef6" path="/var/lib/kubelet/pods/82bf629f-aa66-4ba6-bf3c-eed126e47ef6/volumes"
Sep 19 01:19:29.315347 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:29.315270    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-rmhw5"
Sep 19 01:19:29.382963 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:29.382933    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-rhbdj"
Sep 19 01:19:29.388160 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:29.388106    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-rhbdj\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="74aaa91b-a7ce-4e2a-ac70-863921e21ed3" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-rhbdj"
Sep 19 01:19:39.384807 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:39.384772    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-sd45d"
Sep 19 01:19:39.385279 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:39.384908    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zzrps"
Sep 19 01:19:39.385684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:39.385666    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-svhp6"
Sep 19 01:19:39.571890 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:39.571859    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-sd45d"]
Sep 19 01:19:39.572062 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:39.572034    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-sd45d" podUID="0065e9b0-1ee9-4405-b9e9-dcde83ffa74b" containerName="manager" containerID="cri-o://b425132b0d795f9e011fde1552ba4d888accd80d710c2d9087063cc8264ad703" gracePeriod=10
Sep 19 01:19:39.843030 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:39.842974    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-sd45d"
Sep 19 01:19:39.915770 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:39.915751    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0065e9b0-1ee9-4405-b9e9-dcde83ffa74b-extensions-socket-volume\") pod \"0065e9b0-1ee9-4405-b9e9-dcde83ffa74b\" (UID: \"0065e9b0-1ee9-4405-b9e9-dcde83ffa74b\") "
Sep 19 01:19:39.915867 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:39.915782    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lw6gs\" (UniqueName: \"kubernetes.io/projected/0065e9b0-1ee9-4405-b9e9-dcde83ffa74b-kube-api-access-lw6gs\") pod \"0065e9b0-1ee9-4405-b9e9-dcde83ffa74b\" (UID: \"0065e9b0-1ee9-4405-b9e9-dcde83ffa74b\") "
Sep 19 01:19:39.916120 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:39.916098    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0065e9b0-1ee9-4405-b9e9-dcde83ffa74b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0065e9b0-1ee9-4405-b9e9-dcde83ffa74b" (UID: "0065e9b0-1ee9-4405-b9e9-dcde83ffa74b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:19:39.917837 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:39.917816    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0065e9b0-1ee9-4405-b9e9-dcde83ffa74b-kube-api-access-lw6gs" (OuterVolumeSpecName: "kube-api-access-lw6gs") pod "0065e9b0-1ee9-4405-b9e9-dcde83ffa74b" (UID: "0065e9b0-1ee9-4405-b9e9-dcde83ffa74b"). InnerVolumeSpecName "kube-api-access-lw6gs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:19:40.016865 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:40.016832    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0065e9b0-1ee9-4405-b9e9-dcde83ffa74b-extensions-socket-volume\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:19:40.016865 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:40.016858    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lw6gs\" (UniqueName: \"kubernetes.io/projected/0065e9b0-1ee9-4405-b9e9-dcde83ffa74b-kube-api-access-lw6gs\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:19:40.465956 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:40.465860    2580 generic.go:356] "Generic (PLEG): container finished" podID="0065e9b0-1ee9-4405-b9e9-dcde83ffa74b" containerID="b425132b0d795f9e011fde1552ba4d888accd80d710c2d9087063cc8264ad703" exitCode=0
Sep 19 01:19:40.465956 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:40.465914    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-sd45d"
Sep 19 01:19:40.465956 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:40.465937    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-sd45d" event={"ID":"0065e9b0-1ee9-4405-b9e9-dcde83ffa74b","Type":"ContainerDied","Data":"b425132b0d795f9e011fde1552ba4d888accd80d710c2d9087063cc8264ad703"}
Sep 19 01:19:40.466434 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:40.466028    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-sd45d" event={"ID":"0065e9b0-1ee9-4405-b9e9-dcde83ffa74b","Type":"ContainerDied","Data":"24309ce2861c6946c4abffd8c45d756b35fc2bd70982e0a9e07eec488ea97637"}
Sep 19 01:19:40.466434 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:40.466055    2580 scope.go:117] "RemoveContainer" containerID="b425132b0d795f9e011fde1552ba4d888accd80d710c2d9087063cc8264ad703"
Sep 19 01:19:40.476163 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:40.476136    2580 scope.go:117] "RemoveContainer" containerID="b425132b0d795f9e011fde1552ba4d888accd80d710c2d9087063cc8264ad703"
Sep 19 01:19:40.476435 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:40.476416    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b425132b0d795f9e011fde1552ba4d888accd80d710c2d9087063cc8264ad703\": container with ID starting with b425132b0d795f9e011fde1552ba4d888accd80d710c2d9087063cc8264ad703 not found: ID does not exist" containerID="b425132b0d795f9e011fde1552ba4d888accd80d710c2d9087063cc8264ad703"
Sep 19 01:19:40.476580 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:40.476442    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b425132b0d795f9e011fde1552ba4d888accd80d710c2d9087063cc8264ad703"} err="failed to get container status \"b425132b0d795f9e011fde1552ba4d888accd80d710c2d9087063cc8264ad703\": rpc error: code = NotFound desc = could not find container \"b425132b0d795f9e011fde1552ba4d888accd80d710c2d9087063cc8264ad703\": container with ID starting with b425132b0d795f9e011fde1552ba4d888accd80d710c2d9087063cc8264ad703 not found: ID does not exist"
Sep 19 01:19:40.518966 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:40.518937    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-sd45d"]
Sep 19 01:19:40.528712 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:40.528686    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-sd45d"]
Sep 19 01:19:41.062831 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:41.062798    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0065e9b0-1ee9-4405-b9e9-dcde83ffa74b" path="/var/lib/kubelet/pods/0065e9b0-1ee9-4405-b9e9-dcde83ffa74b/volumes"
Sep 19 01:19:44.428846 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:44.428810    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-bfb8b856-s8hkr" podUID="cdaf3f4e-5b9f-43a5-898d-a27646a0a9df" containerName="console" containerID="cri-o://50115a5f4296ffdeafa5b71bffe4e0c755b774e2787ed7f218a7c567658a037a" gracePeriod=15
Sep 19 01:19:44.679361 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:44.679297    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:19:44.752396 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:44.752369    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-console-oauth-config\") pod \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") "
Sep 19 01:19:44.752573 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:44.752409    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-trusted-ca-bundle\") pod \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") "
Sep 19 01:19:44.752573 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:44.752443    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-console-serving-cert\") pod \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") "
Sep 19 01:19:44.752573 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:44.752481    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-service-ca\") pod \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") "
Sep 19 01:19:44.752573 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:44.752548    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-console-config\") pod \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") "
Sep 19 01:19:44.752801 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:44.752631    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-oauth-serving-cert\") pod \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") "
Sep 19 01:19:44.752801 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:44.752674    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nxwp\" (UniqueName: \"kubernetes.io/projected/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-kube-api-access-6nxwp\") pod \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\" (UID: \"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df\") "
Sep 19 01:19:44.752961 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:44.752936    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-console-config" (OuterVolumeSpecName: "console-config") pod "cdaf3f4e-5b9f-43a5-898d-a27646a0a9df" (UID: "cdaf3f4e-5b9f-43a5-898d-a27646a0a9df"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:19:44.753049 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:44.752965    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-service-ca" (OuterVolumeSpecName: "service-ca") pod "cdaf3f4e-5b9f-43a5-898d-a27646a0a9df" (UID: "cdaf3f4e-5b9f-43a5-898d-a27646a0a9df"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:19:44.753049 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:44.752975    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cdaf3f4e-5b9f-43a5-898d-a27646a0a9df" (UID: "cdaf3f4e-5b9f-43a5-898d-a27646a0a9df"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:19:44.753049 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:44.752981    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cdaf3f4e-5b9f-43a5-898d-a27646a0a9df" (UID: "cdaf3f4e-5b9f-43a5-898d-a27646a0a9df"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:19:44.753049 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:44.753028    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-console-config\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:19:44.754671 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:44.754648    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cdaf3f4e-5b9f-43a5-898d-a27646a0a9df" (UID: "cdaf3f4e-5b9f-43a5-898d-a27646a0a9df"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:19:44.754775 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:44.754742    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cdaf3f4e-5b9f-43a5-898d-a27646a0a9df" (UID: "cdaf3f4e-5b9f-43a5-898d-a27646a0a9df"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:19:44.754849 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:44.754830    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-kube-api-access-6nxwp" (OuterVolumeSpecName: "kube-api-access-6nxwp") pod "cdaf3f4e-5b9f-43a5-898d-a27646a0a9df" (UID: "cdaf3f4e-5b9f-43a5-898d-a27646a0a9df"). InnerVolumeSpecName "kube-api-access-6nxwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:19:44.853428 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:44.853402    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-service-ca\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:19:44.853428 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:44.853425    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-oauth-serving-cert\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:19:44.853428 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:44.853434    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nxwp\" (UniqueName: \"kubernetes.io/projected/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-kube-api-access-6nxwp\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:19:44.853656 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:44.853444    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-console-oauth-config\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:19:44.853656 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:44.853464    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-trusted-ca-bundle\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:19:44.853656 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:44.853473    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df-console-serving-cert\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:19:45.498275 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:45.498235    2580 generic.go:356] "Generic (PLEG): container finished" podID="cdaf3f4e-5b9f-43a5-898d-a27646a0a9df" containerID="50115a5f4296ffdeafa5b71bffe4e0c755b774e2787ed7f218a7c567658a037a" exitCode=0
Sep 19 01:19:45.498760 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:45.498304    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bfb8b856-s8hkr"
Sep 19 01:19:45.498760 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:45.498324    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bfb8b856-s8hkr" event={"ID":"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df","Type":"ContainerDied","Data":"50115a5f4296ffdeafa5b71bffe4e0c755b774e2787ed7f218a7c567658a037a"}
Sep 19 01:19:45.498760 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:45.498402    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bfb8b856-s8hkr" event={"ID":"cdaf3f4e-5b9f-43a5-898d-a27646a0a9df","Type":"ContainerDied","Data":"4f6949f0c3bf5c8e307e006fc6708af5ac3d8f8c026cb66f7f2373262928b100"}
Sep 19 01:19:45.498760 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:45.498423    2580 scope.go:117] "RemoveContainer" containerID="50115a5f4296ffdeafa5b71bffe4e0c755b774e2787ed7f218a7c567658a037a"
Sep 19 01:19:45.507509 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:45.507483    2580 scope.go:117] "RemoveContainer" containerID="50115a5f4296ffdeafa5b71bffe4e0c755b774e2787ed7f218a7c567658a037a"
Sep 19 01:19:45.507769 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:19:45.507748    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50115a5f4296ffdeafa5b71bffe4e0c755b774e2787ed7f218a7c567658a037a\": container with ID starting with 50115a5f4296ffdeafa5b71bffe4e0c755b774e2787ed7f218a7c567658a037a not found: ID does not exist" containerID="50115a5f4296ffdeafa5b71bffe4e0c755b774e2787ed7f218a7c567658a037a"
Sep 19 01:19:45.507835 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:45.507775    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50115a5f4296ffdeafa5b71bffe4e0c755b774e2787ed7f218a7c567658a037a"} err="failed to get container status \"50115a5f4296ffdeafa5b71bffe4e0c755b774e2787ed7f218a7c567658a037a\": rpc error: code = NotFound desc = could not find container \"50115a5f4296ffdeafa5b71bffe4e0c755b774e2787ed7f218a7c567658a037a\": container with ID starting with 50115a5f4296ffdeafa5b71bffe4e0c755b774e2787ed7f218a7c567658a037a not found: ID does not exist"
Sep 19 01:19:45.521061 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:45.521039    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bfb8b856-s8hkr"]
Sep 19 01:19:45.526064 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:45.526042    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bfb8b856-s8hkr"]
Sep 19 01:19:47.064177 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:47.064145    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdaf3f4e-5b9f-43a5-898d-a27646a0a9df" path="/var/lib/kubelet/pods/cdaf3f4e-5b9f-43a5-898d-a27646a0a9df/volumes"
Sep 19 01:19:58.138821 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:58.138786    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-z6vpx"]
Sep 19 01:19:58.139239 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:58.139227    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdaf3f4e-5b9f-43a5-898d-a27646a0a9df" containerName="console"
Sep 19 01:19:58.139239 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:58.139241    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdaf3f4e-5b9f-43a5-898d-a27646a0a9df" containerName="console"
Sep 19 01:19:58.139316 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:58.139260    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0065e9b0-1ee9-4405-b9e9-dcde83ffa74b" containerName="manager"
Sep 19 01:19:58.139316 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:58.139265    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0065e9b0-1ee9-4405-b9e9-dcde83ffa74b" containerName="manager"
Sep 19 01:19:58.139379 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:58.139325    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdaf3f4e-5b9f-43a5-898d-a27646a0a9df" containerName="console"
Sep 19 01:19:58.139379 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:58.139334    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0065e9b0-1ee9-4405-b9e9-dcde83ffa74b" containerName="manager"
Sep 19 01:19:58.143594 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:58.143574    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-z6vpx"
Sep 19 01:19:58.145794 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:58.145772    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 19 01:19:58.145926 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:58.145772    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-p297g\""
Sep 19 01:19:58.153079 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:58.153053    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-z6vpx"]
Sep 19 01:19:58.239295 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:58.239260    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-z6vpx"]
Sep 19 01:19:58.266597 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:58.266567    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/eba730c5-5954-4c76-b102-c510615ce360-config-file\") pod \"limitador-limitador-8588749977-z6vpx\" (UID: \"eba730c5-5954-4c76-b102-c510615ce360\") " pod="kuadrant-system/limitador-limitador-8588749977-z6vpx"
Sep 19 01:19:58.266761 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:58.266605    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96k92\" (UniqueName: \"kubernetes.io/projected/eba730c5-5954-4c76-b102-c510615ce360-kube-api-access-96k92\") pod \"limitador-limitador-8588749977-z6vpx\" (UID: \"eba730c5-5954-4c76-b102-c510615ce360\") " pod="kuadrant-system/limitador-limitador-8588749977-z6vpx"
Sep 19 01:19:58.367946 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:58.367908    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/eba730c5-5954-4c76-b102-c510615ce360-config-file\") pod \"limitador-limitador-8588749977-z6vpx\" (UID: \"eba730c5-5954-4c76-b102-c510615ce360\") " pod="kuadrant-system/limitador-limitador-8588749977-z6vpx"
Sep 19 01:19:58.367946 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:58.367955    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96k92\" (UniqueName: \"kubernetes.io/projected/eba730c5-5954-4c76-b102-c510615ce360-kube-api-access-96k92\") pod \"limitador-limitador-8588749977-z6vpx\" (UID: \"eba730c5-5954-4c76-b102-c510615ce360\") " pod="kuadrant-system/limitador-limitador-8588749977-z6vpx"
Sep 19 01:19:58.368597 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:58.368577    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/eba730c5-5954-4c76-b102-c510615ce360-config-file\") pod \"limitador-limitador-8588749977-z6vpx\" (UID: \"eba730c5-5954-4c76-b102-c510615ce360\") " pod="kuadrant-system/limitador-limitador-8588749977-z6vpx"
Sep 19 01:19:58.377093 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:58.377067    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96k92\" (UniqueName: \"kubernetes.io/projected/eba730c5-5954-4c76-b102-c510615ce360-kube-api-access-96k92\") pod \"limitador-limitador-8588749977-z6vpx\" (UID: \"eba730c5-5954-4c76-b102-c510615ce360\") " pod="kuadrant-system/limitador-limitador-8588749977-z6vpx"
Sep 19 01:19:58.455032 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:58.454958    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-z6vpx"
Sep 19 01:19:58.730886 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:58.730859    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-z6vpx"]
Sep 19 01:19:58.732554 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:19:58.732523    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeba730c5_5954_4c76_b102_c510615ce360.slice/crio-5c311471ca9bd53313e9ade9fae9f1850cd8bdc1defbb3403f3d75727f98125c WatchSource:0}: Error finding container 5c311471ca9bd53313e9ade9fae9f1850cd8bdc1defbb3403f3d75727f98125c: Status 404 returned error can't find the container with id 5c311471ca9bd53313e9ade9fae9f1850cd8bdc1defbb3403f3d75727f98125c
Sep 19 01:19:58.959890 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:58.959847    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-2l6lz"]
Sep 19 01:19:58.964717 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:58.964687    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-2l6lz"
Sep 19 01:19:58.967325 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:58.967297    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-m97cq\""
Sep 19 01:19:58.969287 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:58.969267    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-2l6lz"]
Sep 19 01:19:59.074557 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:59.074491    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6gcq\" (UniqueName: \"kubernetes.io/projected/46e55939-2d07-419c-9001-deaaea5b8be1-kube-api-access-n6gcq\") pod \"authorino-7bcfd8f9cd-2l6lz\" (UID: \"46e55939-2d07-419c-9001-deaaea5b8be1\") " pod="kuadrant-system/authorino-7bcfd8f9cd-2l6lz"
Sep 19 01:19:59.175425 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:59.175390    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6gcq\" (UniqueName: \"kubernetes.io/projected/46e55939-2d07-419c-9001-deaaea5b8be1-kube-api-access-n6gcq\") pod \"authorino-7bcfd8f9cd-2l6lz\" (UID: \"46e55939-2d07-419c-9001-deaaea5b8be1\") " pod="kuadrant-system/authorino-7bcfd8f9cd-2l6lz"
Sep 19 01:19:59.207517 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:59.207483    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6gcq\" (UniqueName: \"kubernetes.io/projected/46e55939-2d07-419c-9001-deaaea5b8be1-kube-api-access-n6gcq\") pod \"authorino-7bcfd8f9cd-2l6lz\" (UID: \"46e55939-2d07-419c-9001-deaaea5b8be1\") " pod="kuadrant-system/authorino-7bcfd8f9cd-2l6lz"
Sep 19 01:19:59.275391 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:59.275362    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-2l6lz"
Sep 19 01:19:59.562136 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:59.562104    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-2l6lz"]
Sep 19 01:19:59.563984 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:19:59.563946    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46e55939_2d07_419c_9001_deaaea5b8be1.slice/crio-84a6837323cda2d61fdaf502d48598f58e319ba80ec9fcb901220d952405b959 WatchSource:0}: Error finding container 84a6837323cda2d61fdaf502d48598f58e319ba80ec9fcb901220d952405b959: Status 404 returned error can't find the container with id 84a6837323cda2d61fdaf502d48598f58e319ba80ec9fcb901220d952405b959
Sep 19 01:19:59.584706 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:59.584639    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-2l6lz" event={"ID":"46e55939-2d07-419c-9001-deaaea5b8be1","Type":"ContainerStarted","Data":"84a6837323cda2d61fdaf502d48598f58e319ba80ec9fcb901220d952405b959"}
Sep 19 01:19:59.586251 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:19:59.586224    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-z6vpx" event={"ID":"eba730c5-5954-4c76-b102-c510615ce360","Type":"ContainerStarted","Data":"5c311471ca9bd53313e9ade9fae9f1850cd8bdc1defbb3403f3d75727f98125c"}
Sep 19 01:20:00.595118 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:00.595065    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-z6vpx" event={"ID":"eba730c5-5954-4c76-b102-c510615ce360","Type":"ContainerStarted","Data":"d8f0acd773c473a3e9149723cde9f6fa7d6207dfddbaa3259b3bff3bfb006867"}
Sep 19 01:20:00.595546 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:00.595287    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-z6vpx"
Sep 19 01:20:00.616608 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:00.616563    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-z6vpx" podStartSLOduration=1.43303697 podStartE2EDuration="2.616547376s" totalImagesPullingTime="1.183510406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:20:00.615670444 +0000 UTC m=+455.522532507" watchObservedRunningTime="2026-09-19 01:20:00.616547376 +0000 UTC m=+455.523409440"
Sep 19 01:20:01.608323 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:01.608286    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-2l6lz" event={"ID":"46e55939-2d07-419c-9001-deaaea5b8be1","Type":"ContainerStarted","Data":"a90048e4e52695fb1d3895fbb918bfb0c24ff456e0627ff0da4adfc6771c409f"}
Sep 19 01:20:01.625354 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:01.625314    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-2l6lz" podStartSLOduration=1.746765075 podStartE2EDuration="3.625300666s" totalImagesPullingTime="1.878535591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:20:01.622762839 +0000 UTC m=+456.529624888" watchObservedRunningTime="2026-09-19 01:20:01.625300666 +0000 UTC m=+456.532162713"
Sep 19 01:20:03.010242 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:03.010202    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-2l6lz"]
Sep 19 01:20:03.620067 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:03.620030    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-2l6lz" podUID="46e55939-2d07-419c-9001-deaaea5b8be1" containerName="authorino" containerID="cri-o://a90048e4e52695fb1d3895fbb918bfb0c24ff456e0627ff0da4adfc6771c409f" gracePeriod=30
Sep 19 01:20:03.865404 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:03.865381    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-2l6lz"
Sep 19 01:20:03.915241 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:03.915173    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6gcq\" (UniqueName: \"kubernetes.io/projected/46e55939-2d07-419c-9001-deaaea5b8be1-kube-api-access-n6gcq\") pod \"46e55939-2d07-419c-9001-deaaea5b8be1\" (UID: \"46e55939-2d07-419c-9001-deaaea5b8be1\") "
Sep 19 01:20:03.917213 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:03.917191    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46e55939-2d07-419c-9001-deaaea5b8be1-kube-api-access-n6gcq" (OuterVolumeSpecName: "kube-api-access-n6gcq") pod "46e55939-2d07-419c-9001-deaaea5b8be1" (UID: "46e55939-2d07-419c-9001-deaaea5b8be1"). InnerVolumeSpecName "kube-api-access-n6gcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:20:04.016599 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:04.016563    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6gcq\" (UniqueName: \"kubernetes.io/projected/46e55939-2d07-419c-9001-deaaea5b8be1-kube-api-access-n6gcq\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:20:04.627389 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:04.627360    2580 generic.go:356] "Generic (PLEG): container finished" podID="46e55939-2d07-419c-9001-deaaea5b8be1" containerID="a90048e4e52695fb1d3895fbb918bfb0c24ff456e0627ff0da4adfc6771c409f" exitCode=0
Sep 19 01:20:04.627604 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:04.627408    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-2l6lz" event={"ID":"46e55939-2d07-419c-9001-deaaea5b8be1","Type":"ContainerDied","Data":"a90048e4e52695fb1d3895fbb918bfb0c24ff456e0627ff0da4adfc6771c409f"}
Sep 19 01:20:04.627604 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:04.627411    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-2l6lz"
Sep 19 01:20:04.627604 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:04.627439    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-2l6lz" event={"ID":"46e55939-2d07-419c-9001-deaaea5b8be1","Type":"ContainerDied","Data":"84a6837323cda2d61fdaf502d48598f58e319ba80ec9fcb901220d952405b959"}
Sep 19 01:20:04.627604 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:04.627456    2580 scope.go:117] "RemoveContainer" containerID="a90048e4e52695fb1d3895fbb918bfb0c24ff456e0627ff0da4adfc6771c409f"
Sep 19 01:20:04.637251 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:04.637234    2580 scope.go:117] "RemoveContainer" containerID="a90048e4e52695fb1d3895fbb918bfb0c24ff456e0627ff0da4adfc6771c409f"
Sep 19 01:20:04.637545 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:20:04.637518    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a90048e4e52695fb1d3895fbb918bfb0c24ff456e0627ff0da4adfc6771c409f\": container with ID starting with a90048e4e52695fb1d3895fbb918bfb0c24ff456e0627ff0da4adfc6771c409f not found: ID does not exist" containerID="a90048e4e52695fb1d3895fbb918bfb0c24ff456e0627ff0da4adfc6771c409f"
Sep 19 01:20:04.637644 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:04.637551    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a90048e4e52695fb1d3895fbb918bfb0c24ff456e0627ff0da4adfc6771c409f"} err="failed to get container status \"a90048e4e52695fb1d3895fbb918bfb0c24ff456e0627ff0da4adfc6771c409f\": rpc error: code = NotFound desc = could not find container \"a90048e4e52695fb1d3895fbb918bfb0c24ff456e0627ff0da4adfc6771c409f\": container with ID starting with a90048e4e52695fb1d3895fbb918bfb0c24ff456e0627ff0da4adfc6771c409f not found: ID does not exist"
Sep 19 01:20:04.649553 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:04.649531    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-2l6lz"]
Sep 19 01:20:04.651848 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:04.651830    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-2l6lz"]
Sep 19 01:20:05.063163 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:05.063130    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46e55939-2d07-419c-9001-deaaea5b8be1" path="/var/lib/kubelet/pods/46e55939-2d07-419c-9001-deaaea5b8be1/volumes"
Sep 19 01:20:11.609855 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:11.609826    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-z6vpx"
Sep 19 01:20:14.029453 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:14.029415    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-z6vpx"]
Sep 19 01:20:14.029848 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:14.029628    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-z6vpx" podUID="eba730c5-5954-4c76-b102-c510615ce360" containerName="limitador" containerID="cri-o://d8f0acd773c473a3e9149723cde9f6fa7d6207dfddbaa3259b3bff3bfb006867" gracePeriod=30
Sep 19 01:20:14.580073 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:14.580050    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-z6vpx"
Sep 19 01:20:14.688687 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:14.688603    2580 generic.go:356] "Generic (PLEG): container finished" podID="eba730c5-5954-4c76-b102-c510615ce360" containerID="d8f0acd773c473a3e9149723cde9f6fa7d6207dfddbaa3259b3bff3bfb006867" exitCode=0
Sep 19 01:20:14.688687 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:14.688658    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-z6vpx"
Sep 19 01:20:14.688861 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:14.688683    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-z6vpx" event={"ID":"eba730c5-5954-4c76-b102-c510615ce360","Type":"ContainerDied","Data":"d8f0acd773c473a3e9149723cde9f6fa7d6207dfddbaa3259b3bff3bfb006867"}
Sep 19 01:20:14.688861 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:14.688767    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-z6vpx" event={"ID":"eba730c5-5954-4c76-b102-c510615ce360","Type":"ContainerDied","Data":"5c311471ca9bd53313e9ade9fae9f1850cd8bdc1defbb3403f3d75727f98125c"}
Sep 19 01:20:14.688861 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:14.688804    2580 scope.go:117] "RemoveContainer" containerID="d8f0acd773c473a3e9149723cde9f6fa7d6207dfddbaa3259b3bff3bfb006867"
Sep 19 01:20:14.697537 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:14.697520    2580 scope.go:117] "RemoveContainer" containerID="d8f0acd773c473a3e9149723cde9f6fa7d6207dfddbaa3259b3bff3bfb006867"
Sep 19 01:20:14.697808 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:20:14.697789    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8f0acd773c473a3e9149723cde9f6fa7d6207dfddbaa3259b3bff3bfb006867\": container with ID starting with d8f0acd773c473a3e9149723cde9f6fa7d6207dfddbaa3259b3bff3bfb006867 not found: ID does not exist" containerID="d8f0acd773c473a3e9149723cde9f6fa7d6207dfddbaa3259b3bff3bfb006867"
Sep 19 01:20:14.697874 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:14.697818    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8f0acd773c473a3e9149723cde9f6fa7d6207dfddbaa3259b3bff3bfb006867"} err="failed to get container status \"d8f0acd773c473a3e9149723cde9f6fa7d6207dfddbaa3259b3bff3bfb006867\": rpc error: code = NotFound desc = could not find container \"d8f0acd773c473a3e9149723cde9f6fa7d6207dfddbaa3259b3bff3bfb006867\": container with ID starting with d8f0acd773c473a3e9149723cde9f6fa7d6207dfddbaa3259b3bff3bfb006867 not found: ID does not exist"
Sep 19 01:20:14.708534 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:14.708515    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96k92\" (UniqueName: \"kubernetes.io/projected/eba730c5-5954-4c76-b102-c510615ce360-kube-api-access-96k92\") pod \"eba730c5-5954-4c76-b102-c510615ce360\" (UID: \"eba730c5-5954-4c76-b102-c510615ce360\") "
Sep 19 01:20:14.708614 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:14.708604    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/eba730c5-5954-4c76-b102-c510615ce360-config-file\") pod \"eba730c5-5954-4c76-b102-c510615ce360\" (UID: \"eba730c5-5954-4c76-b102-c510615ce360\") "
Sep 19 01:20:14.708931 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:14.708902    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eba730c5-5954-4c76-b102-c510615ce360-config-file" (OuterVolumeSpecName: "config-file") pod "eba730c5-5954-4c76-b102-c510615ce360" (UID: "eba730c5-5954-4c76-b102-c510615ce360"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:20:14.710392 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:14.710370    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eba730c5-5954-4c76-b102-c510615ce360-kube-api-access-96k92" (OuterVolumeSpecName: "kube-api-access-96k92") pod "eba730c5-5954-4c76-b102-c510615ce360" (UID: "eba730c5-5954-4c76-b102-c510615ce360"). InnerVolumeSpecName "kube-api-access-96k92". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:20:14.810023 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:14.809976    2580 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/eba730c5-5954-4c76-b102-c510615ce360-config-file\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:20:14.810023 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:14.810019    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96k92\" (UniqueName: \"kubernetes.io/projected/eba730c5-5954-4c76-b102-c510615ce360-kube-api-access-96k92\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:20:15.013331 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:15.013304    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-z6vpx"]
Sep 19 01:20:15.016707 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:15.016684    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-z6vpx"]
Sep 19 01:20:15.063540 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:15.063487    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eba730c5-5954-4c76-b102-c510615ce360" path="/var/lib/kubelet/pods/eba730c5-5954-4c76-b102-c510615ce360/volumes"
Sep 19 01:20:19.918409 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:19.918375    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fcpqw"]
Sep 19 01:20:19.919038 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:19.919018    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eba730c5-5954-4c76-b102-c510615ce360" containerName="limitador"
Sep 19 01:20:19.919120 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:19.919042    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="eba730c5-5954-4c76-b102-c510615ce360" containerName="limitador"
Sep 19 01:20:19.919120 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:19.919065    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46e55939-2d07-419c-9001-deaaea5b8be1" containerName="authorino"
Sep 19 01:20:19.919120 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:19.919074    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="46e55939-2d07-419c-9001-deaaea5b8be1" containerName="authorino"
Sep 19 01:20:19.919274 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:19.919186    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="eba730c5-5954-4c76-b102-c510615ce360" containerName="limitador"
Sep 19 01:20:19.919274 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:19.919206    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="46e55939-2d07-419c-9001-deaaea5b8be1" containerName="authorino"
Sep 19 01:20:19.923652 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:19.923631    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fcpqw"
Sep 19 01:20:19.927660 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:19.927635    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 19 01:20:19.927781 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:19.927758    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-h2rtj\""
Sep 19 01:20:19.928798 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:19.928779    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 19 01:20:19.928894 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:19.928785    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 01:20:19.951696 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:19.951676    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fcpqw"]
Sep 19 01:20:20.056234 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:20.056205    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrgv6\" (UniqueName: \"kubernetes.io/projected/aaf5f974-69f5-43a7-99c9-52f66932c8db-kube-api-access-zrgv6\") pod \"postgres-69fb4594cf-fcpqw\" (UID: \"aaf5f974-69f5-43a7-99c9-52f66932c8db\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fcpqw"
Sep 19 01:20:20.056417 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:20.056242    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/aaf5f974-69f5-43a7-99c9-52f66932c8db-data\") pod \"postgres-69fb4594cf-fcpqw\" (UID: \"aaf5f974-69f5-43a7-99c9-52f66932c8db\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fcpqw"
Sep 19 01:20:20.157663 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:20.157623    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrgv6\" (UniqueName: \"kubernetes.io/projected/aaf5f974-69f5-43a7-99c9-52f66932c8db-kube-api-access-zrgv6\") pod \"postgres-69fb4594cf-fcpqw\" (UID: \"aaf5f974-69f5-43a7-99c9-52f66932c8db\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fcpqw"
Sep 19 01:20:20.157864 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:20.157675    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/aaf5f974-69f5-43a7-99c9-52f66932c8db-data\") pod \"postgres-69fb4594cf-fcpqw\" (UID: \"aaf5f974-69f5-43a7-99c9-52f66932c8db\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fcpqw"
Sep 19 01:20:20.158060 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:20.158039    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/aaf5f974-69f5-43a7-99c9-52f66932c8db-data\") pod \"postgres-69fb4594cf-fcpqw\" (UID: \"aaf5f974-69f5-43a7-99c9-52f66932c8db\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fcpqw"
Sep 19 01:20:20.176733 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:20.176676    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrgv6\" (UniqueName: \"kubernetes.io/projected/aaf5f974-69f5-43a7-99c9-52f66932c8db-kube-api-access-zrgv6\") pod \"postgres-69fb4594cf-fcpqw\" (UID: \"aaf5f974-69f5-43a7-99c9-52f66932c8db\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fcpqw"
Sep 19 01:20:20.233333 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:20.233306    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fcpqw"
Sep 19 01:20:20.516796 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:20.516770    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fcpqw"]
Sep 19 01:20:20.519092 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:20:20.519065    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaf5f974_69f5_43a7_99c9_52f66932c8db.slice/crio-57127561447af50f0e424d6f1744c8f599e8b130b34e12c9986e939fdfd8d3d7 WatchSource:0}: Error finding container 57127561447af50f0e424d6f1744c8f599e8b130b34e12c9986e939fdfd8d3d7: Status 404 returned error can't find the container with id 57127561447af50f0e424d6f1744c8f599e8b130b34e12c9986e939fdfd8d3d7
Sep 19 01:20:20.726737 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:20.726706    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fcpqw" event={"ID":"aaf5f974-69f5-43a7-99c9-52f66932c8db","Type":"ContainerStarted","Data":"57127561447af50f0e424d6f1744c8f599e8b130b34e12c9986e939fdfd8d3d7"}
Sep 19 01:20:26.778787 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:26.778749    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fcpqw" event={"ID":"aaf5f974-69f5-43a7-99c9-52f66932c8db","Type":"ContainerStarted","Data":"7349bcaab9a631f76d7449fd89632dbd964a98530a764f7a8d649506e4e2a105"}
Sep 19 01:20:26.779218 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:26.778870    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fcpqw"
Sep 19 01:20:26.802600 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:26.802555    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fcpqw" podStartSLOduration=1.652639975 podStartE2EDuration="7.802540738s" totalImagesPullingTime="6.149900763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:20:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:20:26.80086495 +0000 UTC m=+481.707726999" watchObservedRunningTime="2026-09-19 01:20:26.802540738 +0000 UTC m=+481.709402786"
Sep 19 01:20:32.813205 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:32.813176    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fcpqw"
Sep 19 01:20:33.364670 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.364636    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-wlml6"]
Sep 19 01:20:33.370755 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.370667    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-wlml6"
Sep 19 01:20:33.372906 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.372883    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-m97cq\""
Sep 19 01:20:33.374209 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.374176    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-wlml6"]
Sep 19 01:20:33.380253 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.380231    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fvbq\" (UniqueName: \"kubernetes.io/projected/873161bb-6ee3-47ae-a3d0-7ee63507e474-kube-api-access-8fvbq\") pod \"authorino-557f876f69-wlml6\" (UID: \"873161bb-6ee3-47ae-a3d0-7ee63507e474\") " pod="kuadrant-system/authorino-557f876f69-wlml6"
Sep 19 01:20:33.481459 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.481428    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fvbq\" (UniqueName: \"kubernetes.io/projected/873161bb-6ee3-47ae-a3d0-7ee63507e474-kube-api-access-8fvbq\") pod \"authorino-557f876f69-wlml6\" (UID: \"873161bb-6ee3-47ae-a3d0-7ee63507e474\") " pod="kuadrant-system/authorino-557f876f69-wlml6"
Sep 19 01:20:33.489418 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.489381    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fvbq\" (UniqueName: \"kubernetes.io/projected/873161bb-6ee3-47ae-a3d0-7ee63507e474-kube-api-access-8fvbq\") pod \"authorino-557f876f69-wlml6\" (UID: \"873161bb-6ee3-47ae-a3d0-7ee63507e474\") " pod="kuadrant-system/authorino-557f876f69-wlml6"
Sep 19 01:20:33.503029 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.503004    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-wlml6"]
Sep 19 01:20:33.503255 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.503242    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-wlml6"
Sep 19 01:20:33.534810 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.534780    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-5md9v"]
Sep 19 01:20:33.540240 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.540220    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-5md9v"
Sep 19 01:20:33.543226 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.543203    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 19 01:20:33.548063 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.548043    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-5md9v"]
Sep 19 01:20:33.582694 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.582629    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clwkk\" (UniqueName: \"kubernetes.io/projected/96e9e6e6-2dd9-4839-9f4e-50bdca348a2d-kube-api-access-clwkk\") pod \"authorino-6d4dfc649f-5md9v\" (UID: \"96e9e6e6-2dd9-4839-9f4e-50bdca348a2d\") " pod="kuadrant-system/authorino-6d4dfc649f-5md9v"
Sep 19 01:20:33.582886 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.582712    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/96e9e6e6-2dd9-4839-9f4e-50bdca348a2d-tls-cert\") pod \"authorino-6d4dfc649f-5md9v\" (UID: \"96e9e6e6-2dd9-4839-9f4e-50bdca348a2d\") " pod="kuadrant-system/authorino-6d4dfc649f-5md9v"
Sep 19 01:20:33.603375 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.603346    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-5md9v"]
Sep 19 01:20:33.603627 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:20:33.603605    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-clwkk tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-5md9v" podUID="96e9e6e6-2dd9-4839-9f4e-50bdca348a2d"
Sep 19 01:20:33.634708 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.634653    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5bd88948bd-vrmt7"]
Sep 19 01:20:33.638843 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.638824    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bd88948bd-vrmt7"
Sep 19 01:20:33.650584 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.650560    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5bd88948bd-vrmt7"]
Sep 19 01:20:33.683655 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.683630    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clwkk\" (UniqueName: \"kubernetes.io/projected/96e9e6e6-2dd9-4839-9f4e-50bdca348a2d-kube-api-access-clwkk\") pod \"authorino-6d4dfc649f-5md9v\" (UID: \"96e9e6e6-2dd9-4839-9f4e-50bdca348a2d\") " pod="kuadrant-system/authorino-6d4dfc649f-5md9v"
Sep 19 01:20:33.683769 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.683687    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/492e04dd-b360-4c0f-8d8d-25fd5f355b39-tls-cert\") pod \"authorino-5bd88948bd-vrmt7\" (UID: \"492e04dd-b360-4c0f-8d8d-25fd5f355b39\") " pod="kuadrant-system/authorino-5bd88948bd-vrmt7"
Sep 19 01:20:33.683769 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.683704    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/96e9e6e6-2dd9-4839-9f4e-50bdca348a2d-tls-cert\") pod \"authorino-6d4dfc649f-5md9v\" (UID: \"96e9e6e6-2dd9-4839-9f4e-50bdca348a2d\") " pod="kuadrant-system/authorino-6d4dfc649f-5md9v"
Sep 19 01:20:33.683847 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.683779    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8kpc\" (UniqueName: \"kubernetes.io/projected/492e04dd-b360-4c0f-8d8d-25fd5f355b39-kube-api-access-s8kpc\") pod \"authorino-5bd88948bd-vrmt7\" (UID: \"492e04dd-b360-4c0f-8d8d-25fd5f355b39\") " pod="kuadrant-system/authorino-5bd88948bd-vrmt7"
Sep 19 01:20:33.685973 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.685948    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/96e9e6e6-2dd9-4839-9f4e-50bdca348a2d-tls-cert\") pod \"authorino-6d4dfc649f-5md9v\" (UID: \"96e9e6e6-2dd9-4839-9f4e-50bdca348a2d\") " pod="kuadrant-system/authorino-6d4dfc649f-5md9v"
Sep 19 01:20:33.694568 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.694552    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clwkk\" (UniqueName: \"kubernetes.io/projected/96e9e6e6-2dd9-4839-9f4e-50bdca348a2d-kube-api-access-clwkk\") pod \"authorino-6d4dfc649f-5md9v\" (UID: \"96e9e6e6-2dd9-4839-9f4e-50bdca348a2d\") " pod="kuadrant-system/authorino-6d4dfc649f-5md9v"
Sep 19 01:20:33.782230 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.782207    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-wlml6"]
Sep 19 01:20:33.783719 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:20:33.783700    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod873161bb_6ee3_47ae_a3d0_7ee63507e474.slice/crio-8f3de6f64b558030f881417da4b848652dde968489c39deb422c871908fd163d WatchSource:0}: Error finding container 8f3de6f64b558030f881417da4b848652dde968489c39deb422c871908fd163d: Status 404 returned error can't find the container with id 8f3de6f64b558030f881417da4b848652dde968489c39deb422c871908fd163d
Sep 19 01:20:33.784248 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.784227    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/492e04dd-b360-4c0f-8d8d-25fd5f355b39-tls-cert\") pod \"authorino-5bd88948bd-vrmt7\" (UID: \"492e04dd-b360-4c0f-8d8d-25fd5f355b39\") " pod="kuadrant-system/authorino-5bd88948bd-vrmt7"
Sep 19 01:20:33.784316 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.784296    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8kpc\" (UniqueName: \"kubernetes.io/projected/492e04dd-b360-4c0f-8d8d-25fd5f355b39-kube-api-access-s8kpc\") pod \"authorino-5bd88948bd-vrmt7\" (UID: \"492e04dd-b360-4c0f-8d8d-25fd5f355b39\") " pod="kuadrant-system/authorino-5bd88948bd-vrmt7"
Sep 19 01:20:33.786471 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.786453    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/492e04dd-b360-4c0f-8d8d-25fd5f355b39-tls-cert\") pod \"authorino-5bd88948bd-vrmt7\" (UID: \"492e04dd-b360-4c0f-8d8d-25fd5f355b39\") " pod="kuadrant-system/authorino-5bd88948bd-vrmt7"
Sep 19 01:20:33.795896 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.795880    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8kpc\" (UniqueName: \"kubernetes.io/projected/492e04dd-b360-4c0f-8d8d-25fd5f355b39-kube-api-access-s8kpc\") pod \"authorino-5bd88948bd-vrmt7\" (UID: \"492e04dd-b360-4c0f-8d8d-25fd5f355b39\") " pod="kuadrant-system/authorino-5bd88948bd-vrmt7"
Sep 19 01:20:33.820539 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.820520    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-5md9v"
Sep 19 01:20:33.820832 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.820540    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-wlml6" event={"ID":"873161bb-6ee3-47ae-a3d0-7ee63507e474","Type":"ContainerStarted","Data":"8f3de6f64b558030f881417da4b848652dde968489c39deb422c871908fd163d"}
Sep 19 01:20:33.829667 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.829646    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-5md9v"
Sep 19 01:20:33.885219 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.885156    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/96e9e6e6-2dd9-4839-9f4e-50bdca348a2d-tls-cert\") pod \"96e9e6e6-2dd9-4839-9f4e-50bdca348a2d\" (UID: \"96e9e6e6-2dd9-4839-9f4e-50bdca348a2d\") "
Sep 19 01:20:33.885219 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.885202    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clwkk\" (UniqueName: \"kubernetes.io/projected/96e9e6e6-2dd9-4839-9f4e-50bdca348a2d-kube-api-access-clwkk\") pod \"96e9e6e6-2dd9-4839-9f4e-50bdca348a2d\" (UID: \"96e9e6e6-2dd9-4839-9f4e-50bdca348a2d\") "
Sep 19 01:20:33.887139 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.887117    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96e9e6e6-2dd9-4839-9f4e-50bdca348a2d-kube-api-access-clwkk" (OuterVolumeSpecName: "kube-api-access-clwkk") pod "96e9e6e6-2dd9-4839-9f4e-50bdca348a2d" (UID: "96e9e6e6-2dd9-4839-9f4e-50bdca348a2d"). InnerVolumeSpecName "kube-api-access-clwkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:20:33.887139 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.887130    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96e9e6e6-2dd9-4839-9f4e-50bdca348a2d-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "96e9e6e6-2dd9-4839-9f4e-50bdca348a2d" (UID: "96e9e6e6-2dd9-4839-9f4e-50bdca348a2d"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:20:33.948453 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.948431    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bd88948bd-vrmt7"
Sep 19 01:20:33.985950 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.985924    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clwkk\" (UniqueName: \"kubernetes.io/projected/96e9e6e6-2dd9-4839-9f4e-50bdca348a2d-kube-api-access-clwkk\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:20:33.985950 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:33.985950    2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/96e9e6e6-2dd9-4839-9f4e-50bdca348a2d-tls-cert\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:20:34.256698 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:20:34.256666    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod492e04dd_b360_4c0f_8d8d_25fd5f355b39.slice/crio-30fb3a71a4176dd04d1da11fe1a783397095fb5d02367342e7658027a99f6c1b WatchSource:0}: Error finding container 30fb3a71a4176dd04d1da11fe1a783397095fb5d02367342e7658027a99f6c1b: Status 404 returned error can't find the container with id 30fb3a71a4176dd04d1da11fe1a783397095fb5d02367342e7658027a99f6c1b
Sep 19 01:20:34.258655 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:34.258633    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5bd88948bd-vrmt7"]
Sep 19 01:20:34.827831 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:34.827796    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bd88948bd-vrmt7" event={"ID":"492e04dd-b360-4c0f-8d8d-25fd5f355b39","Type":"ContainerStarted","Data":"fba8941022a90049deaad93e45a472324b9091f06208066daa609e33d52dae3c"}
Sep 19 01:20:34.827831 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:34.827837    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bd88948bd-vrmt7" event={"ID":"492e04dd-b360-4c0f-8d8d-25fd5f355b39","Type":"ContainerStarted","Data":"30fb3a71a4176dd04d1da11fe1a783397095fb5d02367342e7658027a99f6c1b"}
Sep 19 01:20:34.829379 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:34.829355    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-557f876f69-wlml6" podUID="873161bb-6ee3-47ae-a3d0-7ee63507e474" containerName="authorino" containerID="cri-o://b61246cd6d35f1f16568fca23738e18c2a895d3b10c9eaa3720605775774d7a9" gracePeriod=30
Sep 19 01:20:34.829451 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:34.829380    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-5md9v"
Sep 19 01:20:34.829451 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:34.829362    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-wlml6" event={"ID":"873161bb-6ee3-47ae-a3d0-7ee63507e474","Type":"ContainerStarted","Data":"b61246cd6d35f1f16568fca23738e18c2a895d3b10c9eaa3720605775774d7a9"}
Sep 19 01:20:34.857268 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:34.857229    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5bd88948bd-vrmt7" podStartSLOduration=1.485079989 podStartE2EDuration="1.857215905s" totalImagesPullingTime="372.135916ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:20:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:20:34.855219036 +0000 UTC m=+489.762081085" watchObservedRunningTime="2026-09-19 01:20:34.857215905 +0000 UTC m=+489.764078021"
Sep 19 01:20:34.897692 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:34.897621    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-5md9v"]
Sep 19 01:20:34.907458 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:34.907434    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-5md9v"]
Sep 19 01:20:35.060783 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:35.060759    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-wlml6"
Sep 19 01:20:35.063922 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:35.063899    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96e9e6e6-2dd9-4839-9f4e-50bdca348a2d" path="/var/lib/kubelet/pods/96e9e6e6-2dd9-4839-9f4e-50bdca348a2d/volumes"
Sep 19 01:20:35.095215 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:35.095189    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fvbq\" (UniqueName: \"kubernetes.io/projected/873161bb-6ee3-47ae-a3d0-7ee63507e474-kube-api-access-8fvbq\") pod \"873161bb-6ee3-47ae-a3d0-7ee63507e474\" (UID: \"873161bb-6ee3-47ae-a3d0-7ee63507e474\") "
Sep 19 01:20:35.097189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:35.097165    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/873161bb-6ee3-47ae-a3d0-7ee63507e474-kube-api-access-8fvbq" (OuterVolumeSpecName: "kube-api-access-8fvbq") pod "873161bb-6ee3-47ae-a3d0-7ee63507e474" (UID: "873161bb-6ee3-47ae-a3d0-7ee63507e474"). InnerVolumeSpecName "kube-api-access-8fvbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:20:35.196428 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:35.196358    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fvbq\" (UniqueName: \"kubernetes.io/projected/873161bb-6ee3-47ae-a3d0-7ee63507e474-kube-api-access-8fvbq\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:20:35.836263 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:35.836183    2580 generic.go:356] "Generic (PLEG): container finished" podID="873161bb-6ee3-47ae-a3d0-7ee63507e474" containerID="b61246cd6d35f1f16568fca23738e18c2a895d3b10c9eaa3720605775774d7a9" exitCode=0
Sep 19 01:20:35.836263 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:35.836231    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-wlml6"
Sep 19 01:20:35.836768 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:35.836271    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-wlml6" event={"ID":"873161bb-6ee3-47ae-a3d0-7ee63507e474","Type":"ContainerDied","Data":"b61246cd6d35f1f16568fca23738e18c2a895d3b10c9eaa3720605775774d7a9"}
Sep 19 01:20:35.836768 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:35.836310    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-wlml6" event={"ID":"873161bb-6ee3-47ae-a3d0-7ee63507e474","Type":"ContainerDied","Data":"8f3de6f64b558030f881417da4b848652dde968489c39deb422c871908fd163d"}
Sep 19 01:20:35.836768 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:35.836329    2580 scope.go:117] "RemoveContainer" containerID="b61246cd6d35f1f16568fca23738e18c2a895d3b10c9eaa3720605775774d7a9"
Sep 19 01:20:35.846369 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:35.846353    2580 scope.go:117] "RemoveContainer" containerID="b61246cd6d35f1f16568fca23738e18c2a895d3b10c9eaa3720605775774d7a9"
Sep 19 01:20:35.846684 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:20:35.846663    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b61246cd6d35f1f16568fca23738e18c2a895d3b10c9eaa3720605775774d7a9\": container with ID starting with b61246cd6d35f1f16568fca23738e18c2a895d3b10c9eaa3720605775774d7a9 not found: ID does not exist" containerID="b61246cd6d35f1f16568fca23738e18c2a895d3b10c9eaa3720605775774d7a9"
Sep 19 01:20:35.846761 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:35.846692    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b61246cd6d35f1f16568fca23738e18c2a895d3b10c9eaa3720605775774d7a9"} err="failed to get container status \"b61246cd6d35f1f16568fca23738e18c2a895d3b10c9eaa3720605775774d7a9\": rpc error: code = NotFound desc = could not find container \"b61246cd6d35f1f16568fca23738e18c2a895d3b10c9eaa3720605775774d7a9\": container with ID starting with b61246cd6d35f1f16568fca23738e18c2a895d3b10c9eaa3720605775774d7a9 not found: ID does not exist"
Sep 19 01:20:35.874990 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:35.874959    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-wlml6"]
Sep 19 01:20:35.905937 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:35.905911    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-wlml6"]
Sep 19 01:20:37.063393 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:37.063363    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="873161bb-6ee3-47ae-a3d0-7ee63507e474" path="/var/lib/kubelet/pods/873161bb-6ee3-47ae-a3d0-7ee63507e474/volumes"
Sep 19 01:20:40.563092 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:40.563058    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5bc8cb64b6-9v4gl"]
Sep 19 01:20:40.563458 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:40.563446    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="873161bb-6ee3-47ae-a3d0-7ee63507e474" containerName="authorino"
Sep 19 01:20:40.563458 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:40.563458    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="873161bb-6ee3-47ae-a3d0-7ee63507e474" containerName="authorino"
Sep 19 01:20:40.563571 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:40.563550    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="873161bb-6ee3-47ae-a3d0-7ee63507e474" containerName="authorino"
Sep 19 01:20:40.569159 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:40.569138    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bc8cb64b6-9v4gl"
Sep 19 01:20:40.572681 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:40.572658    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 19 01:20:40.572765 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:40.572705    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 19 01:20:40.572806 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:40.572707    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-v6j6j\""
Sep 19 01:20:40.573134 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:40.573115    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 19 01:20:40.579780 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:40.579758    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bc8cb64b6-9v4gl"]
Sep 19 01:20:40.643993 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:40.643969    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed9df944-2204-4102-9b29-9bfb381fdab7-webhook-cert\") pod \"maas-controller-5bc8cb64b6-9v4gl\" (UID: \"ed9df944-2204-4102-9b29-9bfb381fdab7\") " pod="opendatahub/maas-controller-5bc8cb64b6-9v4gl"
Sep 19 01:20:40.644116 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:40.644039    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlvnp\" (UniqueName: \"kubernetes.io/projected/ed9df944-2204-4102-9b29-9bfb381fdab7-kube-api-access-hlvnp\") pod \"maas-controller-5bc8cb64b6-9v4gl\" (UID: \"ed9df944-2204-4102-9b29-9bfb381fdab7\") " pod="opendatahub/maas-controller-5bc8cb64b6-9v4gl"
Sep 19 01:20:40.644116 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:40.644082    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed9df944-2204-4102-9b29-9bfb381fdab7-metrics-certs\") pod \"maas-controller-5bc8cb64b6-9v4gl\" (UID: \"ed9df944-2204-4102-9b29-9bfb381fdab7\") " pod="opendatahub/maas-controller-5bc8cb64b6-9v4gl"
Sep 19 01:20:40.744749 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:40.744722    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlvnp\" (UniqueName: \"kubernetes.io/projected/ed9df944-2204-4102-9b29-9bfb381fdab7-kube-api-access-hlvnp\") pod \"maas-controller-5bc8cb64b6-9v4gl\" (UID: \"ed9df944-2204-4102-9b29-9bfb381fdab7\") " pod="opendatahub/maas-controller-5bc8cb64b6-9v4gl"
Sep 19 01:20:40.744879 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:40.744759    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed9df944-2204-4102-9b29-9bfb381fdab7-metrics-certs\") pod \"maas-controller-5bc8cb64b6-9v4gl\" (UID: \"ed9df944-2204-4102-9b29-9bfb381fdab7\") " pod="opendatahub/maas-controller-5bc8cb64b6-9v4gl"
Sep 19 01:20:40.744879 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:40.744791    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed9df944-2204-4102-9b29-9bfb381fdab7-webhook-cert\") pod \"maas-controller-5bc8cb64b6-9v4gl\" (UID: \"ed9df944-2204-4102-9b29-9bfb381fdab7\") " pod="opendatahub/maas-controller-5bc8cb64b6-9v4gl"
Sep 19 01:20:40.747276 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:40.747247    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed9df944-2204-4102-9b29-9bfb381fdab7-webhook-cert\") pod \"maas-controller-5bc8cb64b6-9v4gl\" (UID: \"ed9df944-2204-4102-9b29-9bfb381fdab7\") " pod="opendatahub/maas-controller-5bc8cb64b6-9v4gl"
Sep 19 01:20:40.747276 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:40.747269    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed9df944-2204-4102-9b29-9bfb381fdab7-metrics-certs\") pod \"maas-controller-5bc8cb64b6-9v4gl\" (UID: \"ed9df944-2204-4102-9b29-9bfb381fdab7\") " pod="opendatahub/maas-controller-5bc8cb64b6-9v4gl"
Sep 19 01:20:40.757301 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:40.757280    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlvnp\" (UniqueName: \"kubernetes.io/projected/ed9df944-2204-4102-9b29-9bfb381fdab7-kube-api-access-hlvnp\") pod \"maas-controller-5bc8cb64b6-9v4gl\" (UID: \"ed9df944-2204-4102-9b29-9bfb381fdab7\") " pod="opendatahub/maas-controller-5bc8cb64b6-9v4gl"
Sep 19 01:20:40.881183 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:40.881098    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bc8cb64b6-9v4gl"
Sep 19 01:20:41.161094 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:41.161071    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bc8cb64b6-9v4gl"]
Sep 19 01:20:41.162469 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:20:41.162440    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded9df944_2204_4102_9b29_9bfb381fdab7.slice/crio-d63487059bb7c5a93a4e53bb4a22421046a5d072456cfdda7b6ffbcd52b09dc2 WatchSource:0}: Error finding container d63487059bb7c5a93a4e53bb4a22421046a5d072456cfdda7b6ffbcd52b09dc2: Status 404 returned error can't find the container with id d63487059bb7c5a93a4e53bb4a22421046a5d072456cfdda7b6ffbcd52b09dc2
Sep 19 01:20:41.878348 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:41.878305    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bc8cb64b6-9v4gl" event={"ID":"ed9df944-2204-4102-9b29-9bfb381fdab7","Type":"ContainerStarted","Data":"d63487059bb7c5a93a4e53bb4a22421046a5d072456cfdda7b6ffbcd52b09dc2"}
Sep 19 01:20:43.892636 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:43.892594    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bc8cb64b6-9v4gl" event={"ID":"ed9df944-2204-4102-9b29-9bfb381fdab7","Type":"ContainerStarted","Data":"8754becaa61e8e3de2c9117dbf0696340809e559772e6b47c39433d697f9d09f"}
Sep 19 01:20:43.893120 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:43.892742    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5bc8cb64b6-9v4gl"
Sep 19 01:20:43.914223 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:43.914177    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5bc8cb64b6-9v4gl" podStartSLOduration=1.628613828 podStartE2EDuration="3.914162806s" totalImagesPullingTime="2.285548978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:20:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:20:43.911945682 +0000 UTC m=+498.818807731" watchObservedRunningTime="2026-09-19 01:20:43.914162806 +0000 UTC m=+498.821024854"
Sep 19 01:20:54.900148 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:20:54.900110    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5bc8cb64b6-9v4gl"
Sep 19 01:21:18.011440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:18.011360    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-bb6qh"]
Sep 19 01:21:18.017784 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:18.017765    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bb6qh"
Sep 19 01:21:18.024701 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:18.024672    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 19 01:21:18.026548 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:18.026330    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-xzmfd\""
Sep 19 01:21:18.029297 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:18.029275    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-bb6qh"]
Sep 19 01:21:18.077646 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:18.077574    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee7c3e3f-b988-4467-82b7-6bda62a1140d-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-bb6qh\" (UID: \"ee7c3e3f-b988-4467-82b7-6bda62a1140d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bb6qh"
Sep 19 01:21:18.077859 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:18.077784    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqmtn\" (UniqueName: \"kubernetes.io/projected/ee7c3e3f-b988-4467-82b7-6bda62a1140d-kube-api-access-cqmtn\") pod \"payload-pre-processing-59fbc47bf8-bb6qh\" (UID: \"ee7c3e3f-b988-4467-82b7-6bda62a1140d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bb6qh"
Sep 19 01:21:18.110613 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:18.108157    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-42wkt"]
Sep 19 01:21:18.117132 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:18.117092    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-42wkt"
Sep 19 01:21:18.126881 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:18.126852    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-42wkt"]
Sep 19 01:21:18.178957 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:18.178917    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee7c3e3f-b988-4467-82b7-6bda62a1140d-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-bb6qh\" (UID: \"ee7c3e3f-b988-4467-82b7-6bda62a1140d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bb6qh"
Sep 19 01:21:18.178957 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:18.178960    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmbws\" (UniqueName: \"kubernetes.io/projected/c9ba5520-238c-4d7b-9266-16a1c0f76e8d-kube-api-access-qmbws\") pod \"payload-processing-5b54b7fbb7-42wkt\" (UID: \"c9ba5520-238c-4d7b-9266-16a1c0f76e8d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-42wkt"
Sep 19 01:21:18.179244 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:18.179024    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9ba5520-238c-4d7b-9266-16a1c0f76e8d-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-42wkt\" (UID: \"c9ba5520-238c-4d7b-9266-16a1c0f76e8d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-42wkt"
Sep 19 01:21:18.179244 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:18.179107    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqmtn\" (UniqueName: \"kubernetes.io/projected/ee7c3e3f-b988-4467-82b7-6bda62a1140d-kube-api-access-cqmtn\") pod \"payload-pre-processing-59fbc47bf8-bb6qh\" (UID: \"ee7c3e3f-b988-4467-82b7-6bda62a1140d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bb6qh"
Sep 19 01:21:18.179642 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:18.179622    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee7c3e3f-b988-4467-82b7-6bda62a1140d-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-bb6qh\" (UID: \"ee7c3e3f-b988-4467-82b7-6bda62a1140d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bb6qh"
Sep 19 01:21:18.190889 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:18.190861    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqmtn\" (UniqueName: \"kubernetes.io/projected/ee7c3e3f-b988-4467-82b7-6bda62a1140d-kube-api-access-cqmtn\") pod \"payload-pre-processing-59fbc47bf8-bb6qh\" (UID: \"ee7c3e3f-b988-4467-82b7-6bda62a1140d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bb6qh"
Sep 19 01:21:18.280092 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:18.280006    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9ba5520-238c-4d7b-9266-16a1c0f76e8d-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-42wkt\" (UID: \"c9ba5520-238c-4d7b-9266-16a1c0f76e8d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-42wkt"
Sep 19 01:21:18.280092 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:18.280086    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmbws\" (UniqueName: \"kubernetes.io/projected/c9ba5520-238c-4d7b-9266-16a1c0f76e8d-kube-api-access-qmbws\") pod \"payload-processing-5b54b7fbb7-42wkt\" (UID: \"c9ba5520-238c-4d7b-9266-16a1c0f76e8d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-42wkt"
Sep 19 01:21:18.281318 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:18.281288    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9ba5520-238c-4d7b-9266-16a1c0f76e8d-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-42wkt\" (UID: \"c9ba5520-238c-4d7b-9266-16a1c0f76e8d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-42wkt"
Sep 19 01:21:18.288760 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:18.288729    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmbws\" (UniqueName: \"kubernetes.io/projected/c9ba5520-238c-4d7b-9266-16a1c0f76e8d-kube-api-access-qmbws\") pod \"payload-processing-5b54b7fbb7-42wkt\" (UID: \"c9ba5520-238c-4d7b-9266-16a1c0f76e8d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-42wkt"
Sep 19 01:21:18.332550 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:18.332481    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bb6qh"
Sep 19 01:21:18.432344 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:18.431216    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-42wkt"
Sep 19 01:21:18.617840 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:18.617807    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-bb6qh"]
Sep 19 01:21:18.619647 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:21:18.619618    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee7c3e3f_b988_4467_82b7_6bda62a1140d.slice/crio-9a90c3a69d04836cf1d922d11774465316dca05258025355939b27c9ed0065ca WatchSource:0}: Error finding container 9a90c3a69d04836cf1d922d11774465316dca05258025355939b27c9ed0065ca: Status 404 returned error can't find the container with id 9a90c3a69d04836cf1d922d11774465316dca05258025355939b27c9ed0065ca
Sep 19 01:21:18.709880 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:18.709855    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-42wkt"]
Sep 19 01:21:18.711260 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:21:18.711237    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9ba5520_238c_4d7b_9266_16a1c0f76e8d.slice/crio-c5be5002592b5d6f264124d26f93a20e505e9ac8c6ddf6dbff14a27551c5a258 WatchSource:0}: Error finding container c5be5002592b5d6f264124d26f93a20e505e9ac8c6ddf6dbff14a27551c5a258: Status 404 returned error can't find the container with id c5be5002592b5d6f264124d26f93a20e505e9ac8c6ddf6dbff14a27551c5a258
Sep 19 01:21:19.101940 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:19.101903    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-42wkt" event={"ID":"c9ba5520-238c-4d7b-9266-16a1c0f76e8d","Type":"ContainerStarted","Data":"c5be5002592b5d6f264124d26f93a20e505e9ac8c6ddf6dbff14a27551c5a258"}
Sep 19 01:21:19.103182 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:19.103159    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bb6qh" event={"ID":"ee7c3e3f-b988-4467-82b7-6bda62a1140d","Type":"ContainerStarted","Data":"9a90c3a69d04836cf1d922d11774465316dca05258025355939b27c9ed0065ca"}
Sep 19 01:21:20.111531 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:20.111479    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bb6qh" event={"ID":"ee7c3e3f-b988-4467-82b7-6bda62a1140d","Type":"ContainerStarted","Data":"d3a61ddd3c0a73d7f138ea0acf56f8153680da6b2ccef1f618745cda39d87deb"}
Sep 19 01:21:20.111968 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:20.111575    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bb6qh"
Sep 19 01:21:20.113376 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:20.113354    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-42wkt" event={"ID":"c9ba5520-238c-4d7b-9266-16a1c0f76e8d","Type":"ContainerStarted","Data":"db897c8347b15f3e26511fe021747562269f5296a53bb06e402fa17df6a485bb"}
Sep 19 01:21:20.113527 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:20.113402    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-42wkt"
Sep 19 01:21:20.146336 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:20.146266    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bb6qh" podStartSLOduration=1.7693605909999999 podStartE2EDuration="3.146244983s" totalImagesPullingTime="1.376884392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:21:20.144789261 +0000 UTC m=+535.051651311" watchObservedRunningTime="2026-09-19 01:21:20.146244983 +0000 UTC m=+535.053107034"
Sep 19 01:21:31.121091 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:31.121056    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bb6qh"
Sep 19 01:21:31.121590 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:31.121292    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-42wkt"
Sep 19 01:21:31.166406 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:21:31.166349    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-42wkt" podStartSLOduration=11.878011284 podStartE2EDuration="13.166334356s" totalImagesPullingTime="1.288323072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:21:20.169008655 +0000 UTC m=+535.075870704" watchObservedRunningTime="2026-09-19 01:21:31.166334356 +0000 UTC m=+546.073196403"
Sep 19 01:22:27.051594 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:22:27.051562    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f6bdh_74c8e2f6-1b01-4bd1-b491-ad407a15a109/console-operator/1.log"
Sep 19 01:22:27.055851 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:22:27.055827    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f6bdh_74c8e2f6-1b01-4bd1-b491-ad407a15a109/console-operator/1.log"
Sep 19 01:24:49.992688 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:49.992655    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-699c784c97-hc98f"]
Sep 19 01:24:49.996639 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:49.996619    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-699c784c97-hc98f"
Sep 19 01:24:50.013508 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:50.013477    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-699c784c97-hc98f"]
Sep 19 01:24:50.080707 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:50.080672    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twgzj\" (UniqueName: \"kubernetes.io/projected/258a7e21-2f52-4701-a11d-d98fa18aa20f-kube-api-access-twgzj\") pod \"authorino-699c784c97-hc98f\" (UID: \"258a7e21-2f52-4701-a11d-d98fa18aa20f\") " pod="kuadrant-system/authorino-699c784c97-hc98f"
Sep 19 01:24:50.080879 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:50.080753    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/258a7e21-2f52-4701-a11d-d98fa18aa20f-tls-cert\") pod \"authorino-699c784c97-hc98f\" (UID: \"258a7e21-2f52-4701-a11d-d98fa18aa20f\") " pod="kuadrant-system/authorino-699c784c97-hc98f"
Sep 19 01:24:50.181719 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:50.181685    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/258a7e21-2f52-4701-a11d-d98fa18aa20f-tls-cert\") pod \"authorino-699c784c97-hc98f\" (UID: \"258a7e21-2f52-4701-a11d-d98fa18aa20f\") " pod="kuadrant-system/authorino-699c784c97-hc98f"
Sep 19 01:24:50.181880 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:50.181728    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twgzj\" (UniqueName: \"kubernetes.io/projected/258a7e21-2f52-4701-a11d-d98fa18aa20f-kube-api-access-twgzj\") pod \"authorino-699c784c97-hc98f\" (UID: \"258a7e21-2f52-4701-a11d-d98fa18aa20f\") " pod="kuadrant-system/authorino-699c784c97-hc98f"
Sep 19 01:24:50.184295 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:50.184272    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/258a7e21-2f52-4701-a11d-d98fa18aa20f-tls-cert\") pod \"authorino-699c784c97-hc98f\" (UID: \"258a7e21-2f52-4701-a11d-d98fa18aa20f\") " pod="kuadrant-system/authorino-699c784c97-hc98f"
Sep 19 01:24:50.190385 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:50.190361    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twgzj\" (UniqueName: \"kubernetes.io/projected/258a7e21-2f52-4701-a11d-d98fa18aa20f-kube-api-access-twgzj\") pod \"authorino-699c784c97-hc98f\" (UID: \"258a7e21-2f52-4701-a11d-d98fa18aa20f\") " pod="kuadrant-system/authorino-699c784c97-hc98f"
Sep 19 01:24:50.306007 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:50.305949    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-699c784c97-hc98f"
Sep 19 01:24:50.577926 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:50.577902    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-699c784c97-hc98f"]
Sep 19 01:24:50.579236 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:24:50.579211    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod258a7e21_2f52_4701_a11d_d98fa18aa20f.slice/crio-b48bb080abe9ca7716266fa99b28dee9a574fe4fdbb525cad67cab72e9dc85a2 WatchSource:0}: Error finding container b48bb080abe9ca7716266fa99b28dee9a574fe4fdbb525cad67cab72e9dc85a2: Status 404 returned error can't find the container with id b48bb080abe9ca7716266fa99b28dee9a574fe4fdbb525cad67cab72e9dc85a2
Sep 19 01:24:50.580649 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:50.580629    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 01:24:51.379057 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:51.379025    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-699c784c97-hc98f" event={"ID":"258a7e21-2f52-4701-a11d-d98fa18aa20f","Type":"ContainerStarted","Data":"b48bb080abe9ca7716266fa99b28dee9a574fe4fdbb525cad67cab72e9dc85a2"}
Sep 19 01:24:52.411472 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:52.411427    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-699c784c97-hc98f" event={"ID":"258a7e21-2f52-4701-a11d-d98fa18aa20f","Type":"ContainerStarted","Data":"840ff88b4fe58e6adf3718a541095375f1bc37d8cf4e5a322c7ae3728c0ae9e2"}
Sep 19 01:24:52.429940 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:52.429873    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-699c784c97-hc98f" podStartSLOduration=2.514655964 podStartE2EDuration="3.429851811s" totalImagesPullingTime="915.195847ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:24:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:24:52.42883851 +0000 UTC m=+747.335700559" watchObservedRunningTime="2026-09-19 01:24:52.429851811 +0000 UTC m=+747.336713860"
Sep 19 01:24:52.476973 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:52.476940    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5bd88948bd-vrmt7"]
Sep 19 01:24:52.477944 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:52.477267    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5bd88948bd-vrmt7" podUID="492e04dd-b360-4c0f-8d8d-25fd5f355b39" containerName="authorino" containerID="cri-o://fba8941022a90049deaad93e45a472324b9091f06208066daa609e33d52dae3c" gracePeriod=30
Sep 19 01:24:52.743313 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:52.743285    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bd88948bd-vrmt7"
Sep 19 01:24:52.817151 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:52.817112    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8kpc\" (UniqueName: \"kubernetes.io/projected/492e04dd-b360-4c0f-8d8d-25fd5f355b39-kube-api-access-s8kpc\") pod \"492e04dd-b360-4c0f-8d8d-25fd5f355b39\" (UID: \"492e04dd-b360-4c0f-8d8d-25fd5f355b39\") "
Sep 19 01:24:52.817151 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:52.817149    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/492e04dd-b360-4c0f-8d8d-25fd5f355b39-tls-cert\") pod \"492e04dd-b360-4c0f-8d8d-25fd5f355b39\" (UID: \"492e04dd-b360-4c0f-8d8d-25fd5f355b39\") "
Sep 19 01:24:52.819173 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:52.819141    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/492e04dd-b360-4c0f-8d8d-25fd5f355b39-kube-api-access-s8kpc" (OuterVolumeSpecName: "kube-api-access-s8kpc") pod "492e04dd-b360-4c0f-8d8d-25fd5f355b39" (UID: "492e04dd-b360-4c0f-8d8d-25fd5f355b39"). InnerVolumeSpecName "kube-api-access-s8kpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:24:52.828348 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:52.828319    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/492e04dd-b360-4c0f-8d8d-25fd5f355b39-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "492e04dd-b360-4c0f-8d8d-25fd5f355b39" (UID: "492e04dd-b360-4c0f-8d8d-25fd5f355b39"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:24:52.917929 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:52.917890    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8kpc\" (UniqueName: \"kubernetes.io/projected/492e04dd-b360-4c0f-8d8d-25fd5f355b39-kube-api-access-s8kpc\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:24:52.917929 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:52.917919    2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/492e04dd-b360-4c0f-8d8d-25fd5f355b39-tls-cert\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:24:53.418779 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:53.418695    2580 generic.go:356] "Generic (PLEG): container finished" podID="492e04dd-b360-4c0f-8d8d-25fd5f355b39" containerID="fba8941022a90049deaad93e45a472324b9091f06208066daa609e33d52dae3c" exitCode=0
Sep 19 01:24:53.418779 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:53.418746    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bd88948bd-vrmt7"
Sep 19 01:24:53.419289 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:53.418787    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bd88948bd-vrmt7" event={"ID":"492e04dd-b360-4c0f-8d8d-25fd5f355b39","Type":"ContainerDied","Data":"fba8941022a90049deaad93e45a472324b9091f06208066daa609e33d52dae3c"}
Sep 19 01:24:53.419289 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:53.418830    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bd88948bd-vrmt7" event={"ID":"492e04dd-b360-4c0f-8d8d-25fd5f355b39","Type":"ContainerDied","Data":"30fb3a71a4176dd04d1da11fe1a783397095fb5d02367342e7658027a99f6c1b"}
Sep 19 01:24:53.419289 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:53.418852    2580 scope.go:117] "RemoveContainer" containerID="fba8941022a90049deaad93e45a472324b9091f06208066daa609e33d52dae3c"
Sep 19 01:24:53.430537 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:53.430519    2580 scope.go:117] "RemoveContainer" containerID="fba8941022a90049deaad93e45a472324b9091f06208066daa609e33d52dae3c"
Sep 19 01:24:53.430817 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:24:53.430795    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fba8941022a90049deaad93e45a472324b9091f06208066daa609e33d52dae3c\": container with ID starting with fba8941022a90049deaad93e45a472324b9091f06208066daa609e33d52dae3c not found: ID does not exist" containerID="fba8941022a90049deaad93e45a472324b9091f06208066daa609e33d52dae3c"
Sep 19 01:24:53.430913 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:53.430823    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fba8941022a90049deaad93e45a472324b9091f06208066daa609e33d52dae3c"} err="failed to get container status \"fba8941022a90049deaad93e45a472324b9091f06208066daa609e33d52dae3c\": rpc error: code = NotFound desc = could not find container \"fba8941022a90049deaad93e45a472324b9091f06208066daa609e33d52dae3c\": container with ID starting with fba8941022a90049deaad93e45a472324b9091f06208066daa609e33d52dae3c not found: ID does not exist"
Sep 19 01:24:53.442534 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:53.442510    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5bd88948bd-vrmt7"]
Sep 19 01:24:53.445866 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:53.445844    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5bd88948bd-vrmt7"]
Sep 19 01:24:55.064087 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:24:55.064048    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="492e04dd-b360-4c0f-8d8d-25fd5f355b39" path="/var/lib/kubelet/pods/492e04dd-b360-4c0f-8d8d-25fd5f355b39/volumes"
Sep 19 01:25:10.754118 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.754084    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"]
Sep 19 01:25:10.754571 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.754535    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="492e04dd-b360-4c0f-8d8d-25fd5f355b39" containerName="authorino"
Sep 19 01:25:10.754571 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.754548    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="492e04dd-b360-4c0f-8d8d-25fd5f355b39" containerName="authorino"
Sep 19 01:25:10.754650 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.754629    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="492e04dd-b360-4c0f-8d8d-25fd5f355b39" containerName="authorino"
Sep 19 01:25:10.759174 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.759145    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.763109 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.763043    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 19 01:25:10.764622 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.764563    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-c4ce82-openshift-default-dockercfg-8pcfv\""
Sep 19 01:25:10.779519 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.779225    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"]
Sep 19 01:25:10.877237 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.877199    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e8e948d-83e1-4202-880b-53611977d6cb-istiod-ca-cert\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.877419 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.877254    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e8e948d-83e1-4202-880b-53611977d6cb-istio-token\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.877419 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.877318    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-istio-data\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.877419 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.877346    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e8e948d-83e1-4202-880b-53611977d6cb-istio-podinfo\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.877419 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.877371    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-workload-socket\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.877419 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.877387    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d956h\" (UniqueName: \"kubernetes.io/projected/3e8e948d-83e1-4202-880b-53611977d6cb-kube-api-access-d956h\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.877734 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.877452    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-istio-envoy\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.877734 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.877491    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-credential-socket\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.877734 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.877529    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-workload-certs\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.978371 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.978343    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e8e948d-83e1-4202-880b-53611977d6cb-istiod-ca-cert\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.978555 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.978389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e8e948d-83e1-4202-880b-53611977d6cb-istio-token\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.978555 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.978416    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-istio-data\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.978555 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.978432    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e8e948d-83e1-4202-880b-53611977d6cb-istio-podinfo\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.978555 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.978464    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-workload-socket\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.978555 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.978488    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d956h\" (UniqueName: \"kubernetes.io/projected/3e8e948d-83e1-4202-880b-53611977d6cb-kube-api-access-d956h\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.978555 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.978553    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-istio-envoy\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.978816 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.978591    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-credential-socket\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.978816 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.978617    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-workload-certs\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.978913 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.978892    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-istio-data\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.979120 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.979091    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-workload-socket\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.979203 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.979122    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-workload-certs\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.979203 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.979180    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-credential-socket\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.979374 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.979353    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e8e948d-83e1-4202-880b-53611977d6cb-istiod-ca-cert\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.981049 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.981025    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-istio-envoy\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.981280 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.981262    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e8e948d-83e1-4202-880b-53611977d6cb-istio-podinfo\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.987453 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.987432    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e8e948d-83e1-4202-880b-53611977d6cb-istio-token\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:10.987871 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:10.987848    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d956h\" (UniqueName: \"kubernetes.io/projected/3e8e948d-83e1-4202-880b-53611977d6cb-kube-api-access-d956h\") pod \"e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") " pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:11.072265 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:11.072193    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:11.356008 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:11.355977    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"]
Sep 19 01:25:11.357291 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:25:11.357260    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e8e948d_83e1_4202_880b_53611977d6cb.slice/crio-54dccf270e649b3fd6b85329acdd03f77087dc9910c99804f9cf193f2538778c WatchSource:0}: Error finding container 54dccf270e649b3fd6b85329acdd03f77087dc9910c99804f9cf193f2538778c: Status 404 returned error can't find the container with id 54dccf270e649b3fd6b85329acdd03f77087dc9910c99804f9cf193f2538778c
Sep 19 01:25:11.525999 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:11.525964    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46" event={"ID":"3e8e948d-83e1-4202-880b-53611977d6cb","Type":"ContainerStarted","Data":"54dccf270e649b3fd6b85329acdd03f77087dc9910c99804f9cf193f2538778c"}
Sep 19 01:25:12.502335 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:12.502294    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7515-621556"]
Sep 19 01:25:12.506166 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:12.506145    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7515-621556"
Sep 19 01:25:12.509062 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:12.509041    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-frkf9\""
Sep 19 01:25:12.518293 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:12.518269    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7515-621556"]
Sep 19 01:25:12.590861 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:12.590826    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcbfx\" (UniqueName: \"kubernetes.io/projected/b335b23c-b2e2-4d6b-b3a2-1be39005320e-kube-api-access-vcbfx\") pod \"test-curl-7515-621556\" (UID: \"b335b23c-b2e2-4d6b-b3a2-1be39005320e\") " pod="opendatahub/test-curl-7515-621556"
Sep 19 01:25:12.691343 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:12.691313    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcbfx\" (UniqueName: \"kubernetes.io/projected/b335b23c-b2e2-4d6b-b3a2-1be39005320e-kube-api-access-vcbfx\") pod \"test-curl-7515-621556\" (UID: \"b335b23c-b2e2-4d6b-b3a2-1be39005320e\") " pod="opendatahub/test-curl-7515-621556"
Sep 19 01:25:12.713374 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:12.713339    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcbfx\" (UniqueName: \"kubernetes.io/projected/b335b23c-b2e2-4d6b-b3a2-1be39005320e-kube-api-access-vcbfx\") pod \"test-curl-7515-621556\" (UID: \"b335b23c-b2e2-4d6b-b3a2-1be39005320e\") " pod="opendatahub/test-curl-7515-621556"
Sep 19 01:25:12.819282 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:12.819197    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7515-621556"
Sep 19 01:25:13.140148 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:13.140102    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7515-621556"]
Sep 19 01:25:13.142363 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:25:13.142332    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb335b23c_b2e2_4d6b_b3a2_1be39005320e.slice/crio-e18d7dbaae30b716ec281b6c1daaef7f19e37a3272ed619efabb9063a1116b73 WatchSource:0}: Error finding container e18d7dbaae30b716ec281b6c1daaef7f19e37a3272ed619efabb9063a1116b73: Status 404 returned error can't find the container with id e18d7dbaae30b716ec281b6c1daaef7f19e37a3272ed619efabb9063a1116b73
Sep 19 01:25:13.542214 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:13.542154    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7515-621556" event={"ID":"b335b23c-b2e2-4d6b-b3a2-1be39005320e","Type":"ContainerStarted","Data":"e18d7dbaae30b716ec281b6c1daaef7f19e37a3272ed619efabb9063a1116b73"}
Sep 19 01:25:14.659671 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:14.659635    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 19 01:25:14.659959 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:14.659707    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 19 01:25:14.659959 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:14.659735    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 19 01:25:15.010517 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:15.010455    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"]
Sep 19 01:25:15.015309 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:15.014925    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"
Sep 19 01:25:15.019866 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:15.019844    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-8e708f-serving-cert\""
Sep 19 01:25:15.020085 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:15.020070    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 19 01:25:15.020258 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:15.020244    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8d4ph\""
Sep 19 01:25:15.047589 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:15.046705    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"]
Sep 19 01:25:15.116824 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:15.116784    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z525t\" (UniqueName: \"kubernetes.io/projected/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-kube-api-access-z525t\") pod \"maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm\" (UID: \"7d5a338f-fcf2-4b78-8f61-81326a40b7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"
Sep 19 01:25:15.116995 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:15.116869    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm\" (UID: \"7d5a338f-fcf2-4b78-8f61-81326a40b7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"
Sep 19 01:25:15.116995 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:15.116986    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-metrics-certs\") pod \"maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm\" (UID: \"7d5a338f-fcf2-4b78-8f61-81326a40b7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"
Sep 19 01:25:15.117126 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:15.117048    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-maas-api-tls\") pod \"maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm\" (UID: \"7d5a338f-fcf2-4b78-8f61-81326a40b7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"
Sep 19 01:25:15.218595 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:15.218557    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z525t\" (UniqueName: \"kubernetes.io/projected/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-kube-api-access-z525t\") pod \"maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm\" (UID: \"7d5a338f-fcf2-4b78-8f61-81326a40b7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"
Sep 19 01:25:15.218775 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:15.218634    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm\" (UID: \"7d5a338f-fcf2-4b78-8f61-81326a40b7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"
Sep 19 01:25:15.218775 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:15.218692    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-metrics-certs\") pod \"maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm\" (UID: \"7d5a338f-fcf2-4b78-8f61-81326a40b7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"
Sep 19 01:25:15.218775 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:15.218742    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-maas-api-tls\") pod \"maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm\" (UID: \"7d5a338f-fcf2-4b78-8f61-81326a40b7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"
Sep 19 01:25:15.218923 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:25:15.218900    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-serving-cert: secret "maas-api-e2e-worker-w0-8e708f-serving-cert" not found
Sep 19 01:25:15.218991 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:25:15.218976    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-maas-api-tls podName:7d5a338f-fcf2-4b78-8f61-81326a40b7a9 nodeName:}" failed. No retries permitted until 2026-09-19 01:25:15.718951719 +0000 UTC m=+770.625813764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-maas-api-tls") pod "maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm" (UID: "7d5a338f-fcf2-4b78-8f61-81326a40b7a9") : secret "maas-api-e2e-worker-w0-8e708f-serving-cert" not found
Sep 19 01:25:15.219739 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:15.219712    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm\" (UID: \"7d5a338f-fcf2-4b78-8f61-81326a40b7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"
Sep 19 01:25:15.222245 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:15.222218    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-metrics-certs\") pod \"maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm\" (UID: \"7d5a338f-fcf2-4b78-8f61-81326a40b7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"
Sep 19 01:25:15.235747 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:15.235719    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z525t\" (UniqueName: \"kubernetes.io/projected/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-kube-api-access-z525t\") pod \"maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm\" (UID: \"7d5a338f-fcf2-4b78-8f61-81326a40b7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"
Sep 19 01:25:15.557856 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:15.557815    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46" event={"ID":"3e8e948d-83e1-4202-880b-53611977d6cb","Type":"ContainerStarted","Data":"daafa008927a902ceaa7611fd129356bc7da5ef95554b873628b20238674758f"}
Sep 19 01:25:15.585804 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:15.585755    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46" podStartSLOduration=2.285774843 podStartE2EDuration="5.585740375s" totalImagesPullingTime="3.299965532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:25:15.583522445 +0000 UTC m=+770.490384489" watchObservedRunningTime="2026-09-19 01:25:15.585740375 +0000 UTC m=+770.492602493"
Sep 19 01:25:15.724446 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:15.724223    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-maas-api-tls\") pod \"maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm\" (UID: \"7d5a338f-fcf2-4b78-8f61-81326a40b7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"
Sep 19 01:25:15.731736 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:15.731685    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-maas-api-tls\") pod \"maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm\" (UID: \"7d5a338f-fcf2-4b78-8f61-81326a40b7a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"
Sep 19 01:25:15.943851 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:15.943766    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"
Sep 19 01:25:16.073146 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:16.073110    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:16.084957 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:16.084929    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:16.124623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:16.124593    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz"]
Sep 19 01:25:16.131928 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:16.131891    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz"
Sep 19 01:25:16.134204 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:16.134179    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-8e708f\""
Sep 19 01:25:16.134877 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:16.134854    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-8e708f-dockercfg-pn794\""
Sep 19 01:25:16.143444 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:16.143423    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz"]
Sep 19 01:25:16.229341 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:16.229305    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgh7j\" (UniqueName: \"kubernetes.io/projected/d1bff2cc-199c-47c3-aacd-db6b0f522a91-kube-api-access-rgh7j\") pod \"payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz\" (UID: \"d1bff2cc-199c-47c3-aacd-db6b0f522a91\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz"
Sep 19 01:25:16.229571 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:16.229372    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1bff2cc-199c-47c3-aacd-db6b0f522a91-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz\" (UID: \"d1bff2cc-199c-47c3-aacd-db6b0f522a91\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz"
Sep 19 01:25:16.330603 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:16.330573    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgh7j\" (UniqueName: \"kubernetes.io/projected/d1bff2cc-199c-47c3-aacd-db6b0f522a91-kube-api-access-rgh7j\") pod \"payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz\" (UID: \"d1bff2cc-199c-47c3-aacd-db6b0f522a91\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz"
Sep 19 01:25:16.330742 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:16.330615    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1bff2cc-199c-47c3-aacd-db6b0f522a91-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz\" (UID: \"d1bff2cc-199c-47c3-aacd-db6b0f522a91\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz"
Sep 19 01:25:16.331373 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:16.331352    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1bff2cc-199c-47c3-aacd-db6b0f522a91-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz\" (UID: \"d1bff2cc-199c-47c3-aacd-db6b0f522a91\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz"
Sep 19 01:25:16.353174 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:16.353145    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgh7j\" (UniqueName: \"kubernetes.io/projected/d1bff2cc-199c-47c3-aacd-db6b0f522a91-kube-api-access-rgh7j\") pod \"payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz\" (UID: \"d1bff2cc-199c-47c3-aacd-db6b0f522a91\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz"
Sep 19 01:25:16.449896 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:16.449820    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz"
Sep 19 01:25:16.467842 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:16.467812    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"]
Sep 19 01:25:16.469338 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:25:16.469312    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d5a338f_fcf2_4b78_8f61_81326a40b7a9.slice/crio-7d74f9c983f22a469675213412b856d3858513eab9f8a035ce7e3a1d83a7a58e WatchSource:0}: Error finding container 7d74f9c983f22a469675213412b856d3858513eab9f8a035ce7e3a1d83a7a58e: Status 404 returned error can't find the container with id 7d74f9c983f22a469675213412b856d3858513eab9f8a035ce7e3a1d83a7a58e
Sep 19 01:25:16.565391 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:16.565353    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7515-621556" event={"ID":"b335b23c-b2e2-4d6b-b3a2-1be39005320e","Type":"ContainerStarted","Data":"36c14401e9a4dbff38f28497d03dcc9e023557cad539c0f95e59504aea288fdf"}
Sep 19 01:25:16.566672 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:16.566648    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm" event={"ID":"7d5a338f-fcf2-4b78-8f61-81326a40b7a9","Type":"ContainerStarted","Data":"7d74f9c983f22a469675213412b856d3858513eab9f8a035ce7e3a1d83a7a58e"}
Sep 19 01:25:16.566909 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:16.566887    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:16.567870 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:16.567851    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:16.605619 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:16.605572    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7515-621556" podStartSLOduration=1.493631717 podStartE2EDuration="4.605556035s" totalImagesPullingTime="3.111924318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:25:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:25:16.604690636 +0000 UTC m=+771.511552683" watchObservedRunningTime="2026-09-19 01:25:16.605556035 +0000 UTC m=+771.512418084"
Sep 19 01:25:16.746080 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:16.746052    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz"]
Sep 19 01:25:16.747447 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:25:16.747417    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1bff2cc_199c_47c3_aacd_db6b0f522a91.slice/crio-ceaf564285760da230853da457f7d4b53af5f8adbf6b0ed5720e17fa387e11af WatchSource:0}: Error finding container ceaf564285760da230853da457f7d4b53af5f8adbf6b0ed5720e17fa387e11af: Status 404 returned error can't find the container with id ceaf564285760da230853da457f7d4b53af5f8adbf6b0ed5720e17fa387e11af
Sep 19 01:25:17.327454 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.327367    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7515-621556"]
Sep 19 01:25:17.338141 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.338111    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7515-621556"]
Sep 19 01:25:17.531710 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.531675    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p"]
Sep 19 01:25:17.532416 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.532388    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b335b23c-b2e2-4d6b-b3a2-1be39005320e" containerName="test-curl-7515-621556"
Sep 19 01:25:17.532416 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.532418    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b335b23c-b2e2-4d6b-b3a2-1be39005320e" containerName="test-curl-7515-621556"
Sep 19 01:25:17.532649 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.532615    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b335b23c-b2e2-4d6b-b3a2-1be39005320e" containerName="test-curl-7515-621556"
Sep 19 01:25:17.536238 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.536217    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p"
Sep 19 01:25:17.538581 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.538559    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-b3b22c\""
Sep 19 01:25:17.542597 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.542437    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e022024c-87f7-4283-b317-232d61243405-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p\" (UID: \"e022024c-87f7-4283-b317-232d61243405\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p"
Sep 19 01:25:17.542597 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.542547    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w85xx\" (UniqueName: \"kubernetes.io/projected/e022024c-87f7-4283-b317-232d61243405-kube-api-access-w85xx\") pod \"payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p\" (UID: \"e022024c-87f7-4283-b317-232d61243405\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p"
Sep 19 01:25:17.546005 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.545918    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p"]
Sep 19 01:25:17.578760 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.578679    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz" event={"ID":"d1bff2cc-199c-47c3-aacd-db6b0f522a91","Type":"ContainerStarted","Data":"86bc4c12df4f5bd60f772f27078d6243fea08f5b6c67cc785ea90f740b60b712"}
Sep 19 01:25:17.579036 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:25:17.579019    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7515-621556" container="test-curl-7515-621556"
Sep 19 01:25:17.579159 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.579144    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz"
Sep 19 01:25:17.579223 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.579181    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz" event={"ID":"d1bff2cc-199c-47c3-aacd-db6b0f522a91","Type":"ContainerStarted","Data":"ceaf564285760da230853da457f7d4b53af5f8adbf6b0ed5720e17fa387e11af"}
Sep 19 01:25:17.601700 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.601659    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz" podStartSLOduration=1.6016463029999999 podStartE2EDuration="1.601646303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:25:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:25:17.599295669 +0000 UTC m=+772.506157731" watchObservedRunningTime="2026-09-19 01:25:17.601646303 +0000 UTC m=+772.508508350"
Sep 19 01:25:17.643966 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.643906    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w85xx\" (UniqueName: \"kubernetes.io/projected/e022024c-87f7-4283-b317-232d61243405-kube-api-access-w85xx\") pod \"payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p\" (UID: \"e022024c-87f7-4283-b317-232d61243405\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p"
Sep 19 01:25:17.644135 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.644047    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e022024c-87f7-4283-b317-232d61243405-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p\" (UID: \"e022024c-87f7-4283-b317-232d61243405\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p"
Sep 19 01:25:17.645054 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.645025    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e022024c-87f7-4283-b317-232d61243405-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p\" (UID: \"e022024c-87f7-4283-b317-232d61243405\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p"
Sep 19 01:25:17.655980 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.655950    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w85xx\" (UniqueName: \"kubernetes.io/projected/e022024c-87f7-4283-b317-232d61243405-kube-api-access-w85xx\") pod \"payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p\" (UID: \"e022024c-87f7-4283-b317-232d61243405\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p"
Sep 19 01:25:17.664755 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.664726    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft"]
Sep 19 01:25:17.670471 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.670448    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft"
Sep 19 01:25:17.673085 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.673066    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-b3b22c-dockercfg-jprl9\""
Sep 19 01:25:17.680643 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.680620    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft"]
Sep 19 01:25:17.744738 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.744699    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hnwf\" (UniqueName: \"kubernetes.io/projected/57685deb-7ca4-4c80-b48c-7b34ba3c9a8a-kube-api-access-4hnwf\") pod \"payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft\" (UID: \"57685deb-7ca4-4c80-b48c-7b34ba3c9a8a\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft"
Sep 19 01:25:17.744927 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.744755    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57685deb-7ca4-4c80-b48c-7b34ba3c9a8a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft\" (UID: \"57685deb-7ca4-4c80-b48c-7b34ba3c9a8a\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft"
Sep 19 01:25:17.846166 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.846080    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hnwf\" (UniqueName: \"kubernetes.io/projected/57685deb-7ca4-4c80-b48c-7b34ba3c9a8a-kube-api-access-4hnwf\") pod \"payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft\" (UID: \"57685deb-7ca4-4c80-b48c-7b34ba3c9a8a\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft"
Sep 19 01:25:17.846541 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.846169    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57685deb-7ca4-4c80-b48c-7b34ba3c9a8a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft\" (UID: \"57685deb-7ca4-4c80-b48c-7b34ba3c9a8a\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft"
Sep 19 01:25:17.847131 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.847104    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57685deb-7ca4-4c80-b48c-7b34ba3c9a8a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft\" (UID: \"57685deb-7ca4-4c80-b48c-7b34ba3c9a8a\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft"
Sep 19 01:25:17.851190 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.851166    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p"
Sep 19 01:25:17.858554 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.858527    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hnwf\" (UniqueName: \"kubernetes.io/projected/57685deb-7ca4-4c80-b48c-7b34ba3c9a8a-kube-api-access-4hnwf\") pod \"payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft\" (UID: \"57685deb-7ca4-4c80-b48c-7b34ba3c9a8a\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft"
Sep 19 01:25:17.985416 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:17.985245    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft"
Sep 19 01:25:18.322598 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:18.322514    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft"]
Sep 19 01:25:18.323786 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:25:18.323755    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57685deb_7ca4_4c80_b48c_7b34ba3c9a8a.slice/crio-cc0cfaa06a2e3069e0733574e46cf618d628cf6dd051a3a7ad8dc9fc34951683 WatchSource:0}: Error finding container cc0cfaa06a2e3069e0733574e46cf618d628cf6dd051a3a7ad8dc9fc34951683: Status 404 returned error can't find the container with id cc0cfaa06a2e3069e0733574e46cf618d628cf6dd051a3a7ad8dc9fc34951683
Sep 19 01:25:18.326552 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:18.326489    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p"]
Sep 19 01:25:18.327633 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:25:18.327611    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode022024c_87f7_4283_b317_232d61243405.slice/crio-6c7aea40141fe98da4226710bdbc4f168c7c3ba6f8d3d02f6350de646ef3be5f WatchSource:0}: Error finding container 6c7aea40141fe98da4226710bdbc4f168c7c3ba6f8d3d02f6350de646ef3be5f: Status 404 returned error can't find the container with id 6c7aea40141fe98da4226710bdbc4f168c7c3ba6f8d3d02f6350de646ef3be5f
Sep 19 01:25:18.586397 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:18.586361    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p" event={"ID":"e022024c-87f7-4283-b317-232d61243405","Type":"ContainerStarted","Data":"4a8acb7691174b45c2265f3ac82a4896bca4e609669aeb99abf079be0c1a037b"}
Sep 19 01:25:18.586604 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:18.586534    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p"
Sep 19 01:25:18.586604 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:18.586580    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p" event={"ID":"e022024c-87f7-4283-b317-232d61243405","Type":"ContainerStarted","Data":"6c7aea40141fe98da4226710bdbc4f168c7c3ba6f8d3d02f6350de646ef3be5f"}
Sep 19 01:25:18.588365 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:18.588332    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft" event={"ID":"57685deb-7ca4-4c80-b48c-7b34ba3c9a8a","Type":"ContainerStarted","Data":"0705613923ac6ea1bfc2fdb1815204699cb8ffa34b3c0883d39e4728ad657333"}
Sep 19 01:25:18.588541 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:18.588379    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft" event={"ID":"57685deb-7ca4-4c80-b48c-7b34ba3c9a8a","Type":"ContainerStarted","Data":"cc0cfaa06a2e3069e0733574e46cf618d628cf6dd051a3a7ad8dc9fc34951683"}
Sep 19 01:25:18.588541 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:18.588438    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft"
Sep 19 01:25:18.590113 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:18.590070    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm" event={"ID":"7d5a338f-fcf2-4b78-8f61-81326a40b7a9","Type":"ContainerStarted","Data":"8cab26403ad4a938a1733bfccb1b615238937aeb5e3d9fb66622b2ee81863043"}
Sep 19 01:25:18.590263 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:18.590157    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7515-621556" podUID="b335b23c-b2e2-4d6b-b3a2-1be39005320e" containerName="test-curl-7515-621556" containerID="cri-o://36c14401e9a4dbff38f28497d03dcc9e023557cad539c0f95e59504aea288fdf" gracePeriod=2
Sep 19 01:25:18.590356 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:18.590339    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"
Sep 19 01:25:18.613562 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:25:18.611603    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7515-621556\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="b335b23c-b2e2-4d6b-b3a2-1be39005320e" pod="opendatahub/test-curl-7515-621556"
Sep 19 01:25:18.639571 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:18.639488    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm" podStartSLOduration=3.012743615 podStartE2EDuration="4.639470049s" totalImagesPullingTime="1.626726434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:25:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:25:18.637827449 +0000 UTC m=+773.544689500" watchObservedRunningTime="2026-09-19 01:25:18.639470049 +0000 UTC m=+773.546332100"
Sep 19 01:25:18.641388 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:18.641349    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p" podStartSLOduration=1.6413391769999999 podStartE2EDuration="1.641339177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:25:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:25:18.609603977 +0000 UTC m=+773.516466020" watchObservedRunningTime="2026-09-19 01:25:18.641339177 +0000 UTC m=+773.548201228"
Sep 19 01:25:18.666360 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:18.666323    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft" podStartSLOduration=1.666309135 podStartE2EDuration="1.666309135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:25:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:25:18.665074398 +0000 UTC m=+773.571936447" watchObservedRunningTime="2026-09-19 01:25:18.666309135 +0000 UTC m=+773.573171183"
Sep 19 01:25:20.785223 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:20.785197    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7515-621556"
Sep 19 01:25:20.877619 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:20.877529    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcbfx\" (UniqueName: \"kubernetes.io/projected/b335b23c-b2e2-4d6b-b3a2-1be39005320e-kube-api-access-vcbfx\") pod \"b335b23c-b2e2-4d6b-b3a2-1be39005320e\" (UID: \"b335b23c-b2e2-4d6b-b3a2-1be39005320e\") "
Sep 19 01:25:20.879728 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:20.879700    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b335b23c-b2e2-4d6b-b3a2-1be39005320e-kube-api-access-vcbfx" (OuterVolumeSpecName: "kube-api-access-vcbfx") pod "b335b23c-b2e2-4d6b-b3a2-1be39005320e" (UID: "b335b23c-b2e2-4d6b-b3a2-1be39005320e"). InnerVolumeSpecName "kube-api-access-vcbfx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:25:20.978420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:20.978386    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcbfx\" (UniqueName: \"kubernetes.io/projected/b335b23c-b2e2-4d6b-b3a2-1be39005320e-kube-api-access-vcbfx\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:25:21.064133 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:21.064100    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b335b23c-b2e2-4d6b-b3a2-1be39005320e" path="/var/lib/kubelet/pods/b335b23c-b2e2-4d6b-b3a2-1be39005320e/volumes"
Sep 19 01:25:21.611060 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:21.611025    2580 generic.go:356] "Generic (PLEG): container finished" podID="b335b23c-b2e2-4d6b-b3a2-1be39005320e" containerID="36c14401e9a4dbff38f28497d03dcc9e023557cad539c0f95e59504aea288fdf" exitCode=137
Sep 19 01:25:21.611241 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:21.611080    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7515-621556"
Sep 19 01:25:21.611241 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:21.611127    2580 scope.go:117] "RemoveContainer" containerID="36c14401e9a4dbff38f28497d03dcc9e023557cad539c0f95e59504aea288fdf"
Sep 19 01:25:21.620412 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:21.620398    2580 scope.go:117] "RemoveContainer" containerID="36c14401e9a4dbff38f28497d03dcc9e023557cad539c0f95e59504aea288fdf"
Sep 19 01:25:21.620749 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:25:21.620730    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36c14401e9a4dbff38f28497d03dcc9e023557cad539c0f95e59504aea288fdf\": container with ID starting with 36c14401e9a4dbff38f28497d03dcc9e023557cad539c0f95e59504aea288fdf not found: ID does not exist" containerID="36c14401e9a4dbff38f28497d03dcc9e023557cad539c0f95e59504aea288fdf"
Sep 19 01:25:21.620830 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:21.620760    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36c14401e9a4dbff38f28497d03dcc9e023557cad539c0f95e59504aea288fdf"} err="failed to get container status \"36c14401e9a4dbff38f28497d03dcc9e023557cad539c0f95e59504aea288fdf\": rpc error: code = NotFound desc = could not find container \"36c14401e9a4dbff38f28497d03dcc9e023557cad539c0f95e59504aea288fdf\": container with ID starting with 36c14401e9a4dbff38f28497d03dcc9e023557cad539c0f95e59504aea288fdf not found: ID does not exist"
Sep 19 01:25:23.130059 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:23.130027    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft"]
Sep 19 01:25:23.135848 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:23.135830    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft"
Sep 19 01:25:23.140513 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:23.140477    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-c4ce82-dockercfg-mncck\""
Sep 19 01:25:23.141115 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:23.141094    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-c4ce82\""
Sep 19 01:25:23.153386 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:23.153365    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft"]
Sep 19 01:25:23.171357 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:23.171333    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn"]
Sep 19 01:25:23.175116 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:23.175098    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn"
Sep 19 01:25:23.197894 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:23.197867    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn"]
Sep 19 01:25:23.199916 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:23.199887    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e0b0a82-a21a-4e31-8409-09c208370b98-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft\" (UID: \"5e0b0a82-a21a-4e31-8409-09c208370b98\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft"
Sep 19 01:25:23.200023 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:23.199988    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfslp\" (UniqueName: \"kubernetes.io/projected/5e0b0a82-a21a-4e31-8409-09c208370b98-kube-api-access-gfslp\") pod \"payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft\" (UID: \"5e0b0a82-a21a-4e31-8409-09c208370b98\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft"
Sep 19 01:25:23.200106 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:23.200085    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-276lz\" (UniqueName: \"kubernetes.io/projected/29965315-e87d-4bdb-aa4b-e8bc7ed5d275-kube-api-access-276lz\") pod \"payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn\" (UID: \"29965315-e87d-4bdb-aa4b-e8bc7ed5d275\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn"
Sep 19 01:25:23.200196 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:23.200180    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29965315-e87d-4bdb-aa4b-e8bc7ed5d275-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn\" (UID: \"29965315-e87d-4bdb-aa4b-e8bc7ed5d275\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn"
Sep 19 01:25:23.300709 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:23.300678    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfslp\" (UniqueName: \"kubernetes.io/projected/5e0b0a82-a21a-4e31-8409-09c208370b98-kube-api-access-gfslp\") pod \"payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft\" (UID: \"5e0b0a82-a21a-4e31-8409-09c208370b98\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft"
Sep 19 01:25:23.300874 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:23.300732    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-276lz\" (UniqueName: \"kubernetes.io/projected/29965315-e87d-4bdb-aa4b-e8bc7ed5d275-kube-api-access-276lz\") pod \"payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn\" (UID: \"29965315-e87d-4bdb-aa4b-e8bc7ed5d275\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn"
Sep 19 01:25:23.300874 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:23.300760    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29965315-e87d-4bdb-aa4b-e8bc7ed5d275-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn\" (UID: \"29965315-e87d-4bdb-aa4b-e8bc7ed5d275\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn"
Sep 19 01:25:23.300874 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:23.300810    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e0b0a82-a21a-4e31-8409-09c208370b98-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft\" (UID: \"5e0b0a82-a21a-4e31-8409-09c208370b98\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft"
Sep 19 01:25:23.301719 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:23.301696    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e0b0a82-a21a-4e31-8409-09c208370b98-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft\" (UID: \"5e0b0a82-a21a-4e31-8409-09c208370b98\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft"
Sep 19 01:25:23.301788 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:23.301696    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29965315-e87d-4bdb-aa4b-e8bc7ed5d275-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn\" (UID: \"29965315-e87d-4bdb-aa4b-e8bc7ed5d275\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn"
Sep 19 01:25:23.315421 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:23.315395    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfslp\" (UniqueName: \"kubernetes.io/projected/5e0b0a82-a21a-4e31-8409-09c208370b98-kube-api-access-gfslp\") pod \"payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft\" (UID: \"5e0b0a82-a21a-4e31-8409-09c208370b98\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft"
Sep 19 01:25:23.321623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:23.321602    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-276lz\" (UniqueName: \"kubernetes.io/projected/29965315-e87d-4bdb-aa4b-e8bc7ed5d275-kube-api-access-276lz\") pod \"payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn\" (UID: \"29965315-e87d-4bdb-aa4b-e8bc7ed5d275\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn"
Sep 19 01:25:23.454064 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:23.453986    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft"
Sep 19 01:25:23.486563 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:23.486533    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn"
Sep 19 01:25:23.771535 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:23.771509    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft"]
Sep 19 01:25:23.772984 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:25:23.772960    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e0b0a82_a21a_4e31_8409_09c208370b98.slice/crio-7eef2f1ef09a4f526e5d01f91dcbb42e61dc8f9cb36164ef3aa6652295107cb7 WatchSource:0}: Error finding container 7eef2f1ef09a4f526e5d01f91dcbb42e61dc8f9cb36164ef3aa6652295107cb7: Status 404 returned error can't find the container with id 7eef2f1ef09a4f526e5d01f91dcbb42e61dc8f9cb36164ef3aa6652295107cb7
Sep 19 01:25:23.798047 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:23.798027    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn"]
Sep 19 01:25:23.804639 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:25:23.804618    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29965315_e87d_4bdb_aa4b_e8bc7ed5d275.slice/crio-1d60c2289530a747f14d772da598c4e5f20f9e8063bb71403f363f2313b41598 WatchSource:0}: Error finding container 1d60c2289530a747f14d772da598c4e5f20f9e8063bb71403f363f2313b41598: Status 404 returned error can't find the container with id 1d60c2289530a747f14d772da598c4e5f20f9e8063bb71403f363f2313b41598
Sep 19 01:25:24.602003 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:24.601973    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"
Sep 19 01:25:24.635211 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:24.635182    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft" event={"ID":"5e0b0a82-a21a-4e31-8409-09c208370b98","Type":"ContainerStarted","Data":"4531bc1f125ee9506fed2df303f4e1cb9088f1d00a4ed79124e871d3f427e832"}
Sep 19 01:25:24.635402 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:24.635246    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft"
Sep 19 01:25:24.635402 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:24.635270    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft" event={"ID":"5e0b0a82-a21a-4e31-8409-09c208370b98","Type":"ContainerStarted","Data":"7eef2f1ef09a4f526e5d01f91dcbb42e61dc8f9cb36164ef3aa6652295107cb7"}
Sep 19 01:25:24.637236 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:24.637172    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn" event={"ID":"29965315-e87d-4bdb-aa4b-e8bc7ed5d275","Type":"ContainerStarted","Data":"e6431e0ed2f9005cb4ce28537e2cc9babb6d62405550b37a223006f1d279d0dd"}
Sep 19 01:25:24.637236 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:24.637231    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn" event={"ID":"29965315-e87d-4bdb-aa4b-e8bc7ed5d275","Type":"ContainerStarted","Data":"1d60c2289530a747f14d772da598c4e5f20f9e8063bb71403f363f2313b41598"}
Sep 19 01:25:24.637434 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:24.637263    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn"
Sep 19 01:25:24.720335 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:24.720266    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft" podStartSLOduration=1.720245111 podStartE2EDuration="1.720245111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:25:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:25:24.718171275 +0000 UTC m=+779.625033323" watchObservedRunningTime="2026-09-19 01:25:24.720245111 +0000 UTC m=+779.627107162"
Sep 19 01:25:24.762270 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:24.762209    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn" podStartSLOduration=1.76214263 podStartE2EDuration="1.76214263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:25:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:25:24.759680825 +0000 UTC m=+779.666542873" watchObservedRunningTime="2026-09-19 01:25:24.76214263 +0000 UTC m=+779.669004678"
Sep 19 01:25:26.046032 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:26.045998    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8"]
Sep 19 01:25:26.049936 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:26.049917    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8"
Sep 19 01:25:26.055492 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:26.055467    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-a6519132\""
Sep 19 01:25:26.065151 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:26.065124    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8"]
Sep 19 01:25:26.130443 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:26.130411    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fz8p\" (UniqueName: \"kubernetes.io/projected/226d0b32-ad7f-47d1-a8ac-734cebcc399e-kube-api-access-7fz8p\") pod \"payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8\" (UID: \"226d0b32-ad7f-47d1-a8ac-734cebcc399e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8"
Sep 19 01:25:26.130626 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:26.130545    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/226d0b32-ad7f-47d1-a8ac-734cebcc399e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8\" (UID: \"226d0b32-ad7f-47d1-a8ac-734cebcc399e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8"
Sep 19 01:25:26.231572 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:26.231540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/226d0b32-ad7f-47d1-a8ac-734cebcc399e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8\" (UID: \"226d0b32-ad7f-47d1-a8ac-734cebcc399e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8"
Sep 19 01:25:26.231742 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:26.231647    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fz8p\" (UniqueName: \"kubernetes.io/projected/226d0b32-ad7f-47d1-a8ac-734cebcc399e-kube-api-access-7fz8p\") pod \"payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8\" (UID: \"226d0b32-ad7f-47d1-a8ac-734cebcc399e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8"
Sep 19 01:25:26.232399 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:26.232379    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/226d0b32-ad7f-47d1-a8ac-734cebcc399e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8\" (UID: \"226d0b32-ad7f-47d1-a8ac-734cebcc399e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8"
Sep 19 01:25:26.243484 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:26.243457    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fz8p\" (UniqueName: \"kubernetes.io/projected/226d0b32-ad7f-47d1-a8ac-734cebcc399e-kube-api-access-7fz8p\") pod \"payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8\" (UID: \"226d0b32-ad7f-47d1-a8ac-734cebcc399e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8"
Sep 19 01:25:26.362686 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:26.362603    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8"
Sep 19 01:25:26.647705 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:25:26.647678    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod226d0b32_ad7f_47d1_a8ac_734cebcc399e.slice/crio-be47d4e2357c2f283e83eb9ba502e540d337fedddbc491f1892f24da6216c290 WatchSource:0}: Error finding container be47d4e2357c2f283e83eb9ba502e540d337fedddbc491f1892f24da6216c290: Status 404 returned error can't find the container with id be47d4e2357c2f283e83eb9ba502e540d337fedddbc491f1892f24da6216c290
Sep 19 01:25:26.648801 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:26.648654    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8"]
Sep 19 01:25:27.660871 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:27.660835    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8" event={"ID":"226d0b32-ad7f-47d1-a8ac-734cebcc399e","Type":"ContainerStarted","Data":"d9d35a2601ac6255b82185377c8037edc04ee9c928947b175083c3c3472338a4"}
Sep 19 01:25:27.661273 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:27.660890    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8" event={"ID":"226d0b32-ad7f-47d1-a8ac-734cebcc399e","Type":"ContainerStarted","Data":"be47d4e2357c2f283e83eb9ba502e540d337fedddbc491f1892f24da6216c290"}
Sep 19 01:25:27.661273 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:27.660917    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8"
Sep 19 01:25:27.680247 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:27.680202    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8" podStartSLOduration=1.680185695 podStartE2EDuration="1.680185695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:25:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:25:27.679473895 +0000 UTC m=+782.586335944" watchObservedRunningTime="2026-09-19 01:25:27.680185695 +0000 UTC m=+782.587047743"
Sep 19 01:25:29.599399 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:29.599368    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz"
Sep 19 01:25:29.599915 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:29.599551    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p"
Sep 19 01:25:29.599915 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:29.599612    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft"
Sep 19 01:25:35.646950 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:35.646913    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn"
Sep 19 01:25:35.647696 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:35.647668    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft"
Sep 19 01:25:38.670659 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:38.670632    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8"
Sep 19 01:25:42.795836 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:42.795756    2580 generic.go:356] "Generic (PLEG): container finished" podID="3e8e948d-83e1-4202-880b-53611977d6cb" containerID="daafa008927a902ceaa7611fd129356bc7da5ef95554b873628b20238674758f" exitCode=0
Sep 19 01:25:42.796243 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:42.795826    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46" event={"ID":"3e8e948d-83e1-4202-880b-53611977d6cb","Type":"ContainerDied","Data":"daafa008927a902ceaa7611fd129356bc7da5ef95554b873628b20238674758f"}
Sep 19 01:25:42.796288 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:42.796261    2580 scope.go:117] "RemoveContainer" containerID="daafa008927a902ceaa7611fd129356bc7da5ef95554b873628b20238674758f"
Sep 19 01:25:42.797347 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:42.797316    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 19 01:25:42.797447 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:42.797402    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 19 01:25:42.797531 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:42.797448    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 19 01:25:43.804028 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:43.803996    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46" event={"ID":"3e8e948d-83e1-4202-880b-53611977d6cb","Type":"ContainerStarted","Data":"9729901154a2ae80afb4e903fa9c4ce8b87002bd37043784b6d771a9b118fed3"}
Sep 19 01:25:44.073177 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:44.073083    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:44.074137 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:44.074116    2580 patch_prober.go:28] interesting pod/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 19 01:25:44.074191 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:44.074165    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46" podUID="3e8e948d-83e1-4202-880b-53611977d6cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 19 01:25:45.072963 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:45.072930    2580 patch_prober.go:28] interesting pod/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 19 01:25:45.073406 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:45.072991    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46" podUID="3e8e948d-83e1-4202-880b-53611977d6cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 19 01:25:46.073404 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:46.073368    2580 patch_prober.go:28] interesting pod/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 19 01:25:46.073904 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:46.073429    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46" podUID="3e8e948d-83e1-4202-880b-53611977d6cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 19 01:25:46.567769 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:46.567732    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:47.072548 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:47.072485    2580 patch_prober.go:28] interesting pod/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 19 01:25:47.072724 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:47.072575    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46" podUID="3e8e948d-83e1-4202-880b-53611977d6cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 19 01:25:47.921127 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:47.921093    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"]
Sep 19 01:25:47.929780 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:47.929754    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"
Sep 19 01:25:47.939523 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:47.938512    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-5a005d-serving-cert\""
Sep 19 01:25:47.947859 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:47.946919    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"]
Sep 19 01:25:48.073085 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.073054    2580 patch_prober.go:28] interesting pod/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 19 01:25:48.073251 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.073116    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46" podUID="3e8e948d-83e1-4202-880b-53611977d6cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 19 01:25:48.075548 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.075526    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p\" (UID: \"918ea7e0-cbe8-4d4e-b1d9-9443339498bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"
Sep 19 01:25:48.075668 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.075649    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6ksn\" (UniqueName: \"kubernetes.io/projected/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-kube-api-access-f6ksn\") pod \"maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p\" (UID: \"918ea7e0-cbe8-4d4e-b1d9-9443339498bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"
Sep 19 01:25:48.075713 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.075689    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-maas-api-tls\") pod \"maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p\" (UID: \"918ea7e0-cbe8-4d4e-b1d9-9443339498bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"
Sep 19 01:25:48.075749 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.075732    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-metrics-certs\") pod \"maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p\" (UID: \"918ea7e0-cbe8-4d4e-b1d9-9443339498bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"
Sep 19 01:25:48.176720 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.176637    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6ksn\" (UniqueName: \"kubernetes.io/projected/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-kube-api-access-f6ksn\") pod \"maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p\" (UID: \"918ea7e0-cbe8-4d4e-b1d9-9443339498bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"
Sep 19 01:25:48.176720 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.176680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-maas-api-tls\") pod \"maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p\" (UID: \"918ea7e0-cbe8-4d4e-b1d9-9443339498bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"
Sep 19 01:25:48.176957 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.176734    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-metrics-certs\") pod \"maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p\" (UID: \"918ea7e0-cbe8-4d4e-b1d9-9443339498bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"
Sep 19 01:25:48.176957 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.176776    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p\" (UID: \"918ea7e0-cbe8-4d4e-b1d9-9443339498bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"
Sep 19 01:25:48.176957 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:25:48.176848    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-serving-cert: secret "maas-api-e2e-inf-b-5a005d-serving-cert" not found
Sep 19 01:25:48.176957 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:25:48.176924    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-maas-api-tls podName:918ea7e0-cbe8-4d4e-b1d9-9443339498bb nodeName:}" failed. No retries permitted until 2026-09-19 01:25:48.67690416 +0000 UTC m=+803.583766206 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-maas-api-tls") pod "maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p" (UID: "918ea7e0-cbe8-4d4e-b1d9-9443339498bb") : secret "maas-api-e2e-inf-b-5a005d-serving-cert" not found
Sep 19 01:25:48.177486 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.177468    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p\" (UID: \"918ea7e0-cbe8-4d4e-b1d9-9443339498bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"
Sep 19 01:25:48.179297 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.179277    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-metrics-certs\") pod \"maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p\" (UID: \"918ea7e0-cbe8-4d4e-b1d9-9443339498bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"
Sep 19 01:25:48.191630 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.191610    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6ksn\" (UniqueName: \"kubernetes.io/projected/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-kube-api-access-f6ksn\") pod \"maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p\" (UID: \"918ea7e0-cbe8-4d4e-b1d9-9443339498bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"
Sep 19 01:25:48.681707 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.681666    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-maas-api-tls\") pod \"maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p\" (UID: \"918ea7e0-cbe8-4d4e-b1d9-9443339498bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"
Sep 19 01:25:48.684331 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.684306    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-maas-api-tls\") pod \"maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p\" (UID: \"918ea7e0-cbe8-4d4e-b1d9-9443339498bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"
Sep 19 01:25:48.762331 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.762287    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm"]
Sep 19 01:25:48.766320 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.766299    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm"
Sep 19 01:25:48.768896 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.768872    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-5a005d-dockercfg-rp2pt\""
Sep 19 01:25:48.769007 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.768893    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-5a005d\""
Sep 19 01:25:48.775886 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.775862    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm"]
Sep 19 01:25:48.856081 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.855724    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"
Sep 19 01:25:48.884045 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.884013    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mvqr\" (UniqueName: \"kubernetes.io/projected/6b4c2de6-e080-4411-a957-9d84d93cb263-kube-api-access-7mvqr\") pod \"payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm\" (UID: \"6b4c2de6-e080-4411-a957-9d84d93cb263\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm"
Sep 19 01:25:48.884187 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.884078    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b4c2de6-e080-4411-a957-9d84d93cb263-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm\" (UID: \"6b4c2de6-e080-4411-a957-9d84d93cb263\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm"
Sep 19 01:25:48.985304 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.985273    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b4c2de6-e080-4411-a957-9d84d93cb263-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm\" (UID: \"6b4c2de6-e080-4411-a957-9d84d93cb263\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm"
Sep 19 01:25:48.985703 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.985363    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mvqr\" (UniqueName: \"kubernetes.io/projected/6b4c2de6-e080-4411-a957-9d84d93cb263-kube-api-access-7mvqr\") pod \"payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm\" (UID: \"6b4c2de6-e080-4411-a957-9d84d93cb263\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm"
Sep 19 01:25:48.986239 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.986221    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b4c2de6-e080-4411-a957-9d84d93cb263-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm\" (UID: \"6b4c2de6-e080-4411-a957-9d84d93cb263\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm"
Sep 19 01:25:48.994306 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:48.994289    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mvqr\" (UniqueName: \"kubernetes.io/projected/6b4c2de6-e080-4411-a957-9d84d93cb263-kube-api-access-7mvqr\") pod \"payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm\" (UID: \"6b4c2de6-e080-4411-a957-9d84d93cb263\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm"
Sep 19 01:25:49.072869 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:49.072838    2580 patch_prober.go:28] interesting pod/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 19 01:25:49.073044 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:49.072886    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46" podUID="3e8e948d-83e1-4202-880b-53611977d6cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 19 01:25:49.079942 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:49.079917    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm"
Sep 19 01:25:49.148520 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:49.148036    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"]
Sep 19 01:25:49.167175 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:25:49.167050    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod918ea7e0_cbe8_4d4e_b1d9_9443339498bb.slice/crio-5ae29c498f98f3d5bfaba488525205c73173f13eda96d375fc903faff1e4b341 WatchSource:0}: Error finding container 5ae29c498f98f3d5bfaba488525205c73173f13eda96d375fc903faff1e4b341: Status 404 returned error can't find the container with id 5ae29c498f98f3d5bfaba488525205c73173f13eda96d375fc903faff1e4b341
Sep 19 01:25:49.391169 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:49.391135    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm"]
Sep 19 01:25:49.392894 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:25:49.392865    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b4c2de6_e080_4411_a957_9d84d93cb263.slice/crio-08e4f55a368202b1be6b80d06e885ba3b75a70d8831086c98aa21bd3ddbdee4e WatchSource:0}: Error finding container 08e4f55a368202b1be6b80d06e885ba3b75a70d8831086c98aa21bd3ddbdee4e: Status 404 returned error can't find the container with id 08e4f55a368202b1be6b80d06e885ba3b75a70d8831086c98aa21bd3ddbdee4e
Sep 19 01:25:49.866893 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:49.866844    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm" event={"ID":"6b4c2de6-e080-4411-a957-9d84d93cb263","Type":"ContainerStarted","Data":"4fdce4a462c2adab77e6fec94ea2e20d04fe188fb380173affc0f7d766fb4ce3"}
Sep 19 01:25:49.867120 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:49.866922    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm" event={"ID":"6b4c2de6-e080-4411-a957-9d84d93cb263","Type":"ContainerStarted","Data":"08e4f55a368202b1be6b80d06e885ba3b75a70d8831086c98aa21bd3ddbdee4e"}
Sep 19 01:25:49.867120 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:49.867039    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm"
Sep 19 01:25:49.868973 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:49.868947    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p" event={"ID":"918ea7e0-cbe8-4d4e-b1d9-9443339498bb","Type":"ContainerStarted","Data":"e3d91112809e9146f721b281b9c5d06199461544d867363f6e157aa39b5dc5bf"}
Sep 19 01:25:49.869115 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:49.868988    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p" event={"ID":"918ea7e0-cbe8-4d4e-b1d9-9443339498bb","Type":"ContainerStarted","Data":"5ae29c498f98f3d5bfaba488525205c73173f13eda96d375fc903faff1e4b341"}
Sep 19 01:25:49.869115 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:49.869022    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"
Sep 19 01:25:49.885949 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:49.885900    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm" podStartSLOduration=1.885884997 podStartE2EDuration="1.885884997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:25:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:25:49.883341087 +0000 UTC m=+804.790203138" watchObservedRunningTime="2026-09-19 01:25:49.885884997 +0000 UTC m=+804.792747045"
Sep 19 01:25:49.911475 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:49.911394    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p" podStartSLOduration=2.421817308 podStartE2EDuration="2.911378382s" totalImagesPullingTime="489.561074ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:25:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:25:49.909703783 +0000 UTC m=+804.816565832" watchObservedRunningTime="2026-09-19 01:25:49.911378382 +0000 UTC m=+804.818240429"
Sep 19 01:25:50.076847 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:50.076813    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:50.078031 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:50.078008    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:25:52.279398 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:52.279358    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8"]
Sep 19 01:25:52.279932 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:52.279635    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8" podUID="226d0b32-ad7f-47d1-a8ac-734cebcc399e" containerName="payload-pre-processing" containerID="cri-o://d9d35a2601ac6255b82185377c8037edc04ee9c928947b175083c3c3472338a4" gracePeriod=30
Sep 19 01:25:52.539001 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:52.538932    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8"
Sep 19 01:25:52.727628 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:52.727596    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fz8p\" (UniqueName: \"kubernetes.io/projected/226d0b32-ad7f-47d1-a8ac-734cebcc399e-kube-api-access-7fz8p\") pod \"226d0b32-ad7f-47d1-a8ac-734cebcc399e\" (UID: \"226d0b32-ad7f-47d1-a8ac-734cebcc399e\") "
Sep 19 01:25:52.727826 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:52.727783    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/226d0b32-ad7f-47d1-a8ac-734cebcc399e-plugins-config-volume\") pod \"226d0b32-ad7f-47d1-a8ac-734cebcc399e\" (UID: \"226d0b32-ad7f-47d1-a8ac-734cebcc399e\") "
Sep 19 01:25:52.728423 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:52.728394    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/226d0b32-ad7f-47d1-a8ac-734cebcc399e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "226d0b32-ad7f-47d1-a8ac-734cebcc399e" (UID: "226d0b32-ad7f-47d1-a8ac-734cebcc399e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:25:52.729924 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:52.729890    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/226d0b32-ad7f-47d1-a8ac-734cebcc399e-kube-api-access-7fz8p" (OuterVolumeSpecName: "kube-api-access-7fz8p") pod "226d0b32-ad7f-47d1-a8ac-734cebcc399e" (UID: "226d0b32-ad7f-47d1-a8ac-734cebcc399e"). InnerVolumeSpecName "kube-api-access-7fz8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:25:52.828937 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:52.828855    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/226d0b32-ad7f-47d1-a8ac-734cebcc399e-plugins-config-volume\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:25:52.828937 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:52.828889    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fz8p\" (UniqueName: \"kubernetes.io/projected/226d0b32-ad7f-47d1-a8ac-734cebcc399e-kube-api-access-7fz8p\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:25:52.889742 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:52.889705    2580 generic.go:356] "Generic (PLEG): container finished" podID="226d0b32-ad7f-47d1-a8ac-734cebcc399e" containerID="d9d35a2601ac6255b82185377c8037edc04ee9c928947b175083c3c3472338a4" exitCode=0
Sep 19 01:25:52.889742 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:52.889743    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8"
Sep 19 01:25:52.889985 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:52.889781    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8" event={"ID":"226d0b32-ad7f-47d1-a8ac-734cebcc399e","Type":"ContainerDied","Data":"d9d35a2601ac6255b82185377c8037edc04ee9c928947b175083c3c3472338a4"}
Sep 19 01:25:52.889985 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:52.889838    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8" event={"ID":"226d0b32-ad7f-47d1-a8ac-734cebcc399e","Type":"ContainerDied","Data":"be47d4e2357c2f283e83eb9ba502e540d337fedddbc491f1892f24da6216c290"}
Sep 19 01:25:52.889985 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:52.889861    2580 scope.go:117] "RemoveContainer" containerID="d9d35a2601ac6255b82185377c8037edc04ee9c928947b175083c3c3472338a4"
Sep 19 01:25:52.900744 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:52.900725    2580 scope.go:117] "RemoveContainer" containerID="d9d35a2601ac6255b82185377c8037edc04ee9c928947b175083c3c3472338a4"
Sep 19 01:25:52.901007 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:25:52.900990    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9d35a2601ac6255b82185377c8037edc04ee9c928947b175083c3c3472338a4\": container with ID starting with d9d35a2601ac6255b82185377c8037edc04ee9c928947b175083c3c3472338a4 not found: ID does not exist" containerID="d9d35a2601ac6255b82185377c8037edc04ee9c928947b175083c3c3472338a4"
Sep 19 01:25:52.901046 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:52.901015    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9d35a2601ac6255b82185377c8037edc04ee9c928947b175083c3c3472338a4"} err="failed to get container status \"d9d35a2601ac6255b82185377c8037edc04ee9c928947b175083c3c3472338a4\": rpc error: code = NotFound desc = could not find container \"d9d35a2601ac6255b82185377c8037edc04ee9c928947b175083c3c3472338a4\": container with ID starting with d9d35a2601ac6255b82185377c8037edc04ee9c928947b175083c3c3472338a4 not found: ID does not exist"
Sep 19 01:25:52.913526 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:52.913480    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8"]
Sep 19 01:25:52.916301 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:52.916279    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a6519132-5765c95578-nmxz8"]
Sep 19 01:25:53.064010 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:53.063977    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="226d0b32-ad7f-47d1-a8ac-734cebcc399e" path="/var/lib/kubelet/pods/226d0b32-ad7f-47d1-a8ac-734cebcc399e/volumes"
Sep 19 01:25:55.881076 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:25:55.881043    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"
Sep 19 01:26:00.878132 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:26:00.878102    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm"
Sep 19 01:26:51.573227 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:26:51.573196    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24"]
Sep 19 01:26:51.573723 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:26:51.573710    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="226d0b32-ad7f-47d1-a8ac-734cebcc399e" containerName="payload-pre-processing"
Sep 19 01:26:51.573771 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:26:51.573725    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="226d0b32-ad7f-47d1-a8ac-734cebcc399e" containerName="payload-pre-processing"
Sep 19 01:26:51.573812 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:26:51.573803    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="226d0b32-ad7f-47d1-a8ac-734cebcc399e" containerName="payload-pre-processing"
Sep 19 01:26:51.577217 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:26:51.577198    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24"
Sep 19 01:26:51.581396 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:26:51.581377    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-f8c7c101-dockercfg-zdtd8\""
Sep 19 01:26:51.581600 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:26:51.581579    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-f8c7c101\""
Sep 19 01:26:51.594787 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:26:51.594755    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24"]
Sep 19 01:26:51.695253 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:26:51.695223    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxlrc\" (UniqueName: \"kubernetes.io/projected/6d2a0fea-b32c-4a1c-ab29-4a130e96efff-kube-api-access-nxlrc\") pod \"payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24\" (UID: \"6d2a0fea-b32c-4a1c-ab29-4a130e96efff\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24"
Sep 19 01:26:51.695428 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:26:51.695328    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d2a0fea-b32c-4a1c-ab29-4a130e96efff-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24\" (UID: \"6d2a0fea-b32c-4a1c-ab29-4a130e96efff\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24"
Sep 19 01:26:51.796290 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:26:51.796255    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxlrc\" (UniqueName: \"kubernetes.io/projected/6d2a0fea-b32c-4a1c-ab29-4a130e96efff-kube-api-access-nxlrc\") pod \"payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24\" (UID: \"6d2a0fea-b32c-4a1c-ab29-4a130e96efff\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24"
Sep 19 01:26:51.796457 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:26:51.796409    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d2a0fea-b32c-4a1c-ab29-4a130e96efff-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24\" (UID: \"6d2a0fea-b32c-4a1c-ab29-4a130e96efff\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24"
Sep 19 01:26:51.797103 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:26:51.797078    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d2a0fea-b32c-4a1c-ab29-4a130e96efff-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24\" (UID: \"6d2a0fea-b32c-4a1c-ab29-4a130e96efff\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24"
Sep 19 01:26:51.806682 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:26:51.806662    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxlrc\" (UniqueName: \"kubernetes.io/projected/6d2a0fea-b32c-4a1c-ab29-4a130e96efff-kube-api-access-nxlrc\") pod \"payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24\" (UID: \"6d2a0fea-b32c-4a1c-ab29-4a130e96efff\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24"
Sep 19 01:26:51.888544 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:26:51.888444    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24"
Sep 19 01:26:52.207349 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:26:52.207326    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24"]
Sep 19 01:26:52.208956 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:26:52.208932    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d2a0fea_b32c_4a1c_ab29_4a130e96efff.slice/crio-f9994d7c7a9629d061b1c387c2cbe096f4935b8845cafeb5ea47048f332fdd80 WatchSource:0}: Error finding container f9994d7c7a9629d061b1c387c2cbe096f4935b8845cafeb5ea47048f332fdd80: Status 404 returned error can't find the container with id f9994d7c7a9629d061b1c387c2cbe096f4935b8845cafeb5ea47048f332fdd80
Sep 19 01:26:52.303923 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:26:52.303891    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24" event={"ID":"6d2a0fea-b32c-4a1c-ab29-4a130e96efff","Type":"ContainerStarted","Data":"f1ff2be597ee3f9df0e85d458476f02e22885fd7e7dde65a659cbd20be402596"}
Sep 19 01:26:52.304081 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:26:52.303951    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24" event={"ID":"6d2a0fea-b32c-4a1c-ab29-4a130e96efff","Type":"ContainerStarted","Data":"f9994d7c7a9629d061b1c387c2cbe096f4935b8845cafeb5ea47048f332fdd80"}
Sep 19 01:26:52.304163 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:26:52.304111    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24"
Sep 19 01:26:52.343764 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:26:52.343719    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24" podStartSLOduration=1.343702727 podStartE2EDuration="1.343702727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:26:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:26:52.343196497 +0000 UTC m=+867.250058544" watchObservedRunningTime="2026-09-19 01:26:52.343702727 +0000 UTC m=+867.250564774"
Sep 19 01:27:03.311890 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:03.311859    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24"
Sep 19 01:27:27.113021 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:27.112945    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f6bdh_74c8e2f6-1b01-4bd1-b491-ad407a15a109/console-operator/1.log"
Sep 19 01:27:27.114092 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:27.114074    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f6bdh_74c8e2f6-1b01-4bd1-b491-ad407a15a109/console-operator/1.log"
Sep 19 01:27:29.067278 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:29.067246    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24"]
Sep 19 01:27:29.067862 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:29.067461    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24" podUID="6d2a0fea-b32c-4a1c-ab29-4a130e96efff" containerName="payload-pre-processing" containerID="cri-o://f1ff2be597ee3f9df0e85d458476f02e22885fd7e7dde65a659cbd20be402596" gracePeriod=30
Sep 19 01:27:29.338028 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:29.337943    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24"
Sep 19 01:27:29.453350 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:29.453317    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxlrc\" (UniqueName: \"kubernetes.io/projected/6d2a0fea-b32c-4a1c-ab29-4a130e96efff-kube-api-access-nxlrc\") pod \"6d2a0fea-b32c-4a1c-ab29-4a130e96efff\" (UID: \"6d2a0fea-b32c-4a1c-ab29-4a130e96efff\") "
Sep 19 01:27:29.453540 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:29.453447    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d2a0fea-b32c-4a1c-ab29-4a130e96efff-plugins-config-volume\") pod \"6d2a0fea-b32c-4a1c-ab29-4a130e96efff\" (UID: \"6d2a0fea-b32c-4a1c-ab29-4a130e96efff\") "
Sep 19 01:27:29.453842 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:29.453817    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d2a0fea-b32c-4a1c-ab29-4a130e96efff-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6d2a0fea-b32c-4a1c-ab29-4a130e96efff" (UID: "6d2a0fea-b32c-4a1c-ab29-4a130e96efff"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:27:29.455474 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:29.455451    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d2a0fea-b32c-4a1c-ab29-4a130e96efff-kube-api-access-nxlrc" (OuterVolumeSpecName: "kube-api-access-nxlrc") pod "6d2a0fea-b32c-4a1c-ab29-4a130e96efff" (UID: "6d2a0fea-b32c-4a1c-ab29-4a130e96efff"). InnerVolumeSpecName "kube-api-access-nxlrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:27:29.554350 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:29.554316    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d2a0fea-b32c-4a1c-ab29-4a130e96efff-plugins-config-volume\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:27:29.554350 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:29.554350    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxlrc\" (UniqueName: \"kubernetes.io/projected/6d2a0fea-b32c-4a1c-ab29-4a130e96efff-kube-api-access-nxlrc\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:27:29.554942 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:29.554918    2580 generic.go:356] "Generic (PLEG): container finished" podID="6d2a0fea-b32c-4a1c-ab29-4a130e96efff" containerID="f1ff2be597ee3f9df0e85d458476f02e22885fd7e7dde65a659cbd20be402596" exitCode=0
Sep 19 01:27:29.555008 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:29.554965    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24"
Sep 19 01:27:29.555063 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:29.555015    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24" event={"ID":"6d2a0fea-b32c-4a1c-ab29-4a130e96efff","Type":"ContainerDied","Data":"f1ff2be597ee3f9df0e85d458476f02e22885fd7e7dde65a659cbd20be402596"}
Sep 19 01:27:29.555119 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:29.555077    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24" event={"ID":"6d2a0fea-b32c-4a1c-ab29-4a130e96efff","Type":"ContainerDied","Data":"f9994d7c7a9629d061b1c387c2cbe096f4935b8845cafeb5ea47048f332fdd80"}
Sep 19 01:27:29.555119 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:29.555111    2580 scope.go:117] "RemoveContainer" containerID="f1ff2be597ee3f9df0e85d458476f02e22885fd7e7dde65a659cbd20be402596"
Sep 19 01:27:29.565185 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:29.565168    2580 scope.go:117] "RemoveContainer" containerID="f1ff2be597ee3f9df0e85d458476f02e22885fd7e7dde65a659cbd20be402596"
Sep 19 01:27:29.565469 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:27:29.565449    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1ff2be597ee3f9df0e85d458476f02e22885fd7e7dde65a659cbd20be402596\": container with ID starting with f1ff2be597ee3f9df0e85d458476f02e22885fd7e7dde65a659cbd20be402596 not found: ID does not exist" containerID="f1ff2be597ee3f9df0e85d458476f02e22885fd7e7dde65a659cbd20be402596"
Sep 19 01:27:29.565563 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:29.565481    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1ff2be597ee3f9df0e85d458476f02e22885fd7e7dde65a659cbd20be402596"} err="failed to get container status \"f1ff2be597ee3f9df0e85d458476f02e22885fd7e7dde65a659cbd20be402596\": rpc error: code = NotFound desc = could not find container \"f1ff2be597ee3f9df0e85d458476f02e22885fd7e7dde65a659cbd20be402596\": container with ID starting with f1ff2be597ee3f9df0e85d458476f02e22885fd7e7dde65a659cbd20be402596 not found: ID does not exist"
Sep 19 01:27:29.576298 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:29.576275    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24"]
Sep 19 01:27:29.580027 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:29.580007    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f8c7c101-857574d9d-87q24"]
Sep 19 01:27:31.065456 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:31.065418    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d2a0fea-b32c-4a1c-ab29-4a130e96efff" path="/var/lib/kubelet/pods/6d2a0fea-b32c-4a1c-ab29-4a130e96efff/volumes"
Sep 19 01:27:39.417740 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.417708    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv"]
Sep 19 01:27:39.418236 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.418222    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d2a0fea-b32c-4a1c-ab29-4a130e96efff" containerName="payload-pre-processing"
Sep 19 01:27:39.418280 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.418238    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d2a0fea-b32c-4a1c-ab29-4a130e96efff" containerName="payload-pre-processing"
Sep 19 01:27:39.418331 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.418322    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d2a0fea-b32c-4a1c-ab29-4a130e96efff" containerName="payload-pre-processing"
Sep 19 01:27:39.423048 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.423029    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9"]
Sep 19 01:27:39.423156 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.423138    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv"
Sep 19 01:27:39.425712 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.425692    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-55340d68\""
Sep 19 01:27:39.425942 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.425795    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-55340d68-dockercfg-24vw8\""
Sep 19 01:27:39.426833 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.426789    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9"
Sep 19 01:27:39.438021 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.437996    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9"]
Sep 19 01:27:39.443539 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.443517    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv"]
Sep 19 01:27:39.554378 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.554344    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24ca7f86-c58b-4996-ab2d-ef94aef5ef92-plugins-config-volume\") pod \"payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv\" (UID: \"24ca7f86-c58b-4996-ab2d-ef94aef5ef92\") " pod="openshift-ingress/payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv"
Sep 19 01:27:39.554585 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.554402    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7638211-a6fe-4e9a-ab6e-8bc21e4111a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9\" (UID: \"a7638211-a6fe-4e9a-ab6e-8bc21e4111a1\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9"
Sep 19 01:27:39.554585 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.554555    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m9tt\" (UniqueName: \"kubernetes.io/projected/24ca7f86-c58b-4996-ab2d-ef94aef5ef92-kube-api-access-9m9tt\") pod \"payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv\" (UID: \"24ca7f86-c58b-4996-ab2d-ef94aef5ef92\") " pod="openshift-ingress/payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv"
Sep 19 01:27:39.554715 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.554590    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5pfd\" (UniqueName: \"kubernetes.io/projected/a7638211-a6fe-4e9a-ab6e-8bc21e4111a1-kube-api-access-s5pfd\") pod \"payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9\" (UID: \"a7638211-a6fe-4e9a-ab6e-8bc21e4111a1\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9"
Sep 19 01:27:39.655700 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.655662    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7638211-a6fe-4e9a-ab6e-8bc21e4111a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9\" (UID: \"a7638211-a6fe-4e9a-ab6e-8bc21e4111a1\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9"
Sep 19 01:27:39.655884 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.655748    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m9tt\" (UniqueName: \"kubernetes.io/projected/24ca7f86-c58b-4996-ab2d-ef94aef5ef92-kube-api-access-9m9tt\") pod \"payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv\" (UID: \"24ca7f86-c58b-4996-ab2d-ef94aef5ef92\") " pod="openshift-ingress/payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv"
Sep 19 01:27:39.655884 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.655773    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5pfd\" (UniqueName: \"kubernetes.io/projected/a7638211-a6fe-4e9a-ab6e-8bc21e4111a1-kube-api-access-s5pfd\") pod \"payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9\" (UID: \"a7638211-a6fe-4e9a-ab6e-8bc21e4111a1\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9"
Sep 19 01:27:39.655884 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.655827    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24ca7f86-c58b-4996-ab2d-ef94aef5ef92-plugins-config-volume\") pod \"payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv\" (UID: \"24ca7f86-c58b-4996-ab2d-ef94aef5ef92\") " pod="openshift-ingress/payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv"
Sep 19 01:27:39.656477 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.656455    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7638211-a6fe-4e9a-ab6e-8bc21e4111a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9\" (UID: \"a7638211-a6fe-4e9a-ab6e-8bc21e4111a1\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9"
Sep 19 01:27:39.656477 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.656469    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24ca7f86-c58b-4996-ab2d-ef94aef5ef92-plugins-config-volume\") pod \"payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv\" (UID: \"24ca7f86-c58b-4996-ab2d-ef94aef5ef92\") " pod="openshift-ingress/payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv"
Sep 19 01:27:39.665090 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.665063    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5pfd\" (UniqueName: \"kubernetes.io/projected/a7638211-a6fe-4e9a-ab6e-8bc21e4111a1-kube-api-access-s5pfd\") pod \"payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9\" (UID: \"a7638211-a6fe-4e9a-ab6e-8bc21e4111a1\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9"
Sep 19 01:27:39.665351 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.665336    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m9tt\" (UniqueName: \"kubernetes.io/projected/24ca7f86-c58b-4996-ab2d-ef94aef5ef92-kube-api-access-9m9tt\") pod \"payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv\" (UID: \"24ca7f86-c58b-4996-ab2d-ef94aef5ef92\") " pod="openshift-ingress/payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv"
Sep 19 01:27:39.737404 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.737380    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv"
Sep 19 01:27:39.745595 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:39.745573    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9"
Sep 19 01:27:40.038956 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:27:40.038925    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7638211_a6fe_4e9a_ab6e_8bc21e4111a1.slice/crio-9b641c2ce29cc1f57c3db5963047dc254933700b6e1cfd81cff6ef6a7fffc44f WatchSource:0}: Error finding container 9b641c2ce29cc1f57c3db5963047dc254933700b6e1cfd81cff6ef6a7fffc44f: Status 404 returned error can't find the container with id 9b641c2ce29cc1f57c3db5963047dc254933700b6e1cfd81cff6ef6a7fffc44f
Sep 19 01:27:40.039605 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:40.039468    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv"]
Sep 19 01:27:40.040187 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:27:40.040161    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24ca7f86_c58b_4996_ab2d_ef94aef5ef92.slice/crio-e523da79cae0200ec02f8d288834cb52a8af6dd604777f86b03804a419e8a538 WatchSource:0}: Error finding container e523da79cae0200ec02f8d288834cb52a8af6dd604777f86b03804a419e8a538: Status 404 returned error can't find the container with id e523da79cae0200ec02f8d288834cb52a8af6dd604777f86b03804a419e8a538
Sep 19 01:27:40.043091 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:40.043072    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9"]
Sep 19 01:27:40.632324 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:40.632283    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9" event={"ID":"a7638211-a6fe-4e9a-ab6e-8bc21e4111a1","Type":"ContainerStarted","Data":"35ba32bd903cfc9cded002d5b590f1d8ab1a3d718e02c8f9f6451b774c77d139"}
Sep 19 01:27:40.632818 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:40.632379    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9" event={"ID":"a7638211-a6fe-4e9a-ab6e-8bc21e4111a1","Type":"ContainerStarted","Data":"9b641c2ce29cc1f57c3db5963047dc254933700b6e1cfd81cff6ef6a7fffc44f"}
Sep 19 01:27:40.632818 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:40.632410    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9"
Sep 19 01:27:40.634209 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:40.634176    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv" event={"ID":"24ca7f86-c58b-4996-ab2d-ef94aef5ef92","Type":"ContainerStarted","Data":"b17f7b81049cb23c2678309f6cfba6748ab79e1b1fd7e32a53fef29e011709c4"}
Sep 19 01:27:40.634339 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:40.634223    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv" event={"ID":"24ca7f86-c58b-4996-ab2d-ef94aef5ef92","Type":"ContainerStarted","Data":"e523da79cae0200ec02f8d288834cb52a8af6dd604777f86b03804a419e8a538"}
Sep 19 01:27:40.634339 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:40.634278    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv"
Sep 19 01:27:40.652032 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:40.651981    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9" podStartSLOduration=1.6519632720000001 podStartE2EDuration="1.651963272s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:27:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:27:40.647531293 +0000 UTC m=+915.554393342" watchObservedRunningTime="2026-09-19 01:27:40.651963272 +0000 UTC m=+915.558825325"
Sep 19 01:27:40.663527 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:40.663473    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv" podStartSLOduration=1.663459364 podStartE2EDuration="1.663459364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:27:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:27:40.663062327 +0000 UTC m=+915.569924377" watchObservedRunningTime="2026-09-19 01:27:40.663459364 +0000 UTC m=+915.570321403"
Sep 19 01:27:51.644568 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:51.644528    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9"
Sep 19 01:27:51.644989 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:27:51.644604    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv"
Sep 19 01:28:19.343785 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.343752    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv"]
Sep 19 01:28:19.344347 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.344002    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv" podUID="24ca7f86-c58b-4996-ab2d-ef94aef5ef92" containerName="payload-processing" containerID="cri-o://b17f7b81049cb23c2678309f6cfba6748ab79e1b1fd7e32a53fef29e011709c4" gracePeriod=30
Sep 19 01:28:19.354665 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.354637    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9"]
Sep 19 01:28:19.354901 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.354870    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9" podUID="a7638211-a6fe-4e9a-ab6e-8bc21e4111a1" containerName="payload-pre-processing" containerID="cri-o://35ba32bd903cfc9cded002d5b590f1d8ab1a3d718e02c8f9f6451b774c77d139" gracePeriod=30
Sep 19 01:28:19.664988 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.664842    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv"
Sep 19 01:28:19.745186 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.745162    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9"
Sep 19 01:28:19.764092 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.764065    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24ca7f86-c58b-4996-ab2d-ef94aef5ef92-plugins-config-volume\") pod \"24ca7f86-c58b-4996-ab2d-ef94aef5ef92\" (UID: \"24ca7f86-c58b-4996-ab2d-ef94aef5ef92\") "
Sep 19 01:28:19.764268 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.764113    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9m9tt\" (UniqueName: \"kubernetes.io/projected/24ca7f86-c58b-4996-ab2d-ef94aef5ef92-kube-api-access-9m9tt\") pod \"24ca7f86-c58b-4996-ab2d-ef94aef5ef92\" (UID: \"24ca7f86-c58b-4996-ab2d-ef94aef5ef92\") "
Sep 19 01:28:19.764473 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.764445    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24ca7f86-c58b-4996-ab2d-ef94aef5ef92-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "24ca7f86-c58b-4996-ab2d-ef94aef5ef92" (UID: "24ca7f86-c58b-4996-ab2d-ef94aef5ef92"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:28:19.766442 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.766415    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24ca7f86-c58b-4996-ab2d-ef94aef5ef92-kube-api-access-9m9tt" (OuterVolumeSpecName: "kube-api-access-9m9tt") pod "24ca7f86-c58b-4996-ab2d-ef94aef5ef92" (UID: "24ca7f86-c58b-4996-ab2d-ef94aef5ef92"). InnerVolumeSpecName "kube-api-access-9m9tt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:28:19.865343 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.865308    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5pfd\" (UniqueName: \"kubernetes.io/projected/a7638211-a6fe-4e9a-ab6e-8bc21e4111a1-kube-api-access-s5pfd\") pod \"a7638211-a6fe-4e9a-ab6e-8bc21e4111a1\" (UID: \"a7638211-a6fe-4e9a-ab6e-8bc21e4111a1\") "
Sep 19 01:28:19.865343 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.865352    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7638211-a6fe-4e9a-ab6e-8bc21e4111a1-plugins-config-volume\") pod \"a7638211-a6fe-4e9a-ab6e-8bc21e4111a1\" (UID: \"a7638211-a6fe-4e9a-ab6e-8bc21e4111a1\") "
Sep 19 01:28:19.865666 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.865654    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24ca7f86-c58b-4996-ab2d-ef94aef5ef92-plugins-config-volume\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:28:19.865712 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.865668    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9m9tt\" (UniqueName: \"kubernetes.io/projected/24ca7f86-c58b-4996-ab2d-ef94aef5ef92-kube-api-access-9m9tt\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:28:19.865754 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.865737    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7638211-a6fe-4e9a-ab6e-8bc21e4111a1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a7638211-a6fe-4e9a-ab6e-8bc21e4111a1" (UID: "a7638211-a6fe-4e9a-ab6e-8bc21e4111a1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:28:19.867421 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.867374    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7638211-a6fe-4e9a-ab6e-8bc21e4111a1-kube-api-access-s5pfd" (OuterVolumeSpecName: "kube-api-access-s5pfd") pod "a7638211-a6fe-4e9a-ab6e-8bc21e4111a1" (UID: "a7638211-a6fe-4e9a-ab6e-8bc21e4111a1"). InnerVolumeSpecName "kube-api-access-s5pfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:28:19.903195 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.903161    2580 generic.go:356] "Generic (PLEG): container finished" podID="a7638211-a6fe-4e9a-ab6e-8bc21e4111a1" containerID="35ba32bd903cfc9cded002d5b590f1d8ab1a3d718e02c8f9f6451b774c77d139" exitCode=0
Sep 19 01:28:19.903335 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.903204    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9"
Sep 19 01:28:19.903335 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.903253    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9" event={"ID":"a7638211-a6fe-4e9a-ab6e-8bc21e4111a1","Type":"ContainerDied","Data":"35ba32bd903cfc9cded002d5b590f1d8ab1a3d718e02c8f9f6451b774c77d139"}
Sep 19 01:28:19.903335 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.903312    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9" event={"ID":"a7638211-a6fe-4e9a-ab6e-8bc21e4111a1","Type":"ContainerDied","Data":"9b641c2ce29cc1f57c3db5963047dc254933700b6e1cfd81cff6ef6a7fffc44f"}
Sep 19 01:28:19.903523 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.903342    2580 scope.go:117] "RemoveContainer" containerID="35ba32bd903cfc9cded002d5b590f1d8ab1a3d718e02c8f9f6451b774c77d139"
Sep 19 01:28:19.904836 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.904801    2580 generic.go:356] "Generic (PLEG): container finished" podID="24ca7f86-c58b-4996-ab2d-ef94aef5ef92" containerID="b17f7b81049cb23c2678309f6cfba6748ab79e1b1fd7e32a53fef29e011709c4" exitCode=0
Sep 19 01:28:19.904959 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.904843    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv"
Sep 19 01:28:19.904959 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.904871    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv" event={"ID":"24ca7f86-c58b-4996-ab2d-ef94aef5ef92","Type":"ContainerDied","Data":"b17f7b81049cb23c2678309f6cfba6748ab79e1b1fd7e32a53fef29e011709c4"}
Sep 19 01:28:19.904959 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.904919    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv" event={"ID":"24ca7f86-c58b-4996-ab2d-ef94aef5ef92","Type":"ContainerDied","Data":"e523da79cae0200ec02f8d288834cb52a8af6dd604777f86b03804a419e8a538"}
Sep 19 01:28:19.915189 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.915173    2580 scope.go:117] "RemoveContainer" containerID="35ba32bd903cfc9cded002d5b590f1d8ab1a3d718e02c8f9f6451b774c77d139"
Sep 19 01:28:19.915444 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:28:19.915428    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35ba32bd903cfc9cded002d5b590f1d8ab1a3d718e02c8f9f6451b774c77d139\": container with ID starting with 35ba32bd903cfc9cded002d5b590f1d8ab1a3d718e02c8f9f6451b774c77d139 not found: ID does not exist" containerID="35ba32bd903cfc9cded002d5b590f1d8ab1a3d718e02c8f9f6451b774c77d139"
Sep 19 01:28:19.915483 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.915454    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35ba32bd903cfc9cded002d5b590f1d8ab1a3d718e02c8f9f6451b774c77d139"} err="failed to get container status \"35ba32bd903cfc9cded002d5b590f1d8ab1a3d718e02c8f9f6451b774c77d139\": rpc error: code = NotFound desc = could not find container \"35ba32bd903cfc9cded002d5b590f1d8ab1a3d718e02c8f9f6451b774c77d139\": container with ID starting with 35ba32bd903cfc9cded002d5b590f1d8ab1a3d718e02c8f9f6451b774c77d139 not found: ID does not exist"
Sep 19 01:28:19.915483 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.915471    2580 scope.go:117] "RemoveContainer" containerID="b17f7b81049cb23c2678309f6cfba6748ab79e1b1fd7e32a53fef29e011709c4"
Sep 19 01:28:19.924159 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.924110    2580 scope.go:117] "RemoveContainer" containerID="b17f7b81049cb23c2678309f6cfba6748ab79e1b1fd7e32a53fef29e011709c4"
Sep 19 01:28:19.924662 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:28:19.924491    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b17f7b81049cb23c2678309f6cfba6748ab79e1b1fd7e32a53fef29e011709c4\": container with ID starting with b17f7b81049cb23c2678309f6cfba6748ab79e1b1fd7e32a53fef29e011709c4 not found: ID does not exist" containerID="b17f7b81049cb23c2678309f6cfba6748ab79e1b1fd7e32a53fef29e011709c4"
Sep 19 01:28:19.924772 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.924671    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b17f7b81049cb23c2678309f6cfba6748ab79e1b1fd7e32a53fef29e011709c4"} err="failed to get container status \"b17f7b81049cb23c2678309f6cfba6748ab79e1b1fd7e32a53fef29e011709c4\": rpc error: code = NotFound desc = could not find container \"b17f7b81049cb23c2678309f6cfba6748ab79e1b1fd7e32a53fef29e011709c4\": container with ID starting with b17f7b81049cb23c2678309f6cfba6748ab79e1b1fd7e32a53fef29e011709c4 not found: ID does not exist"
Sep 19 01:28:19.931762 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.931726    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9"]
Sep 19 01:28:19.935878 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.935856    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-55340d68-68b8d8bfc8-5ndq9"]
Sep 19 01:28:19.942855 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.942833    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv"]
Sep 19 01:28:19.946884 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.946863    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-55340d68-67b48c7677-kpcwv"]
Sep 19 01:28:19.966781 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.966758    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5pfd\" (UniqueName: \"kubernetes.io/projected/a7638211-a6fe-4e9a-ab6e-8bc21e4111a1-kube-api-access-s5pfd\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:28:19.966781 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:19.966778    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7638211-a6fe-4e9a-ab6e-8bc21e4111a1-plugins-config-volume\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:28:21.064767 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:21.064725    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24ca7f86-c58b-4996-ab2d-ef94aef5ef92" path="/var/lib/kubelet/pods/24ca7f86-c58b-4996-ab2d-ef94aef5ef92/volumes"
Sep 19 01:28:21.065309 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:21.065287    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7638211-a6fe-4e9a-ab6e-8bc21e4111a1" path="/var/lib/kubelet/pods/a7638211-a6fe-4e9a-ab6e-8bc21e4111a1/volumes"
Sep 19 01:28:51.416322 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:51.416239    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft"]
Sep 19 01:28:51.416893 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:51.416532    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft" podUID="57685deb-7ca4-4c80-b48c-7b34ba3c9a8a" containerName="payload-processing" containerID="cri-o://0705613923ac6ea1bfc2fdb1815204699cb8ffa34b3c0883d39e4728ad657333" gracePeriod=30
Sep 19 01:28:51.421849 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:51.421822    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p"]
Sep 19 01:28:51.422220 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:51.422192    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p" podUID="e022024c-87f7-4283-b317-232d61243405" containerName="payload-pre-processing" containerID="cri-o://4a8acb7691174b45c2265f3ac82a4896bca4e609669aeb99abf079be0c1a037b" gracePeriod=30
Sep 19 01:28:51.692865 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:51.692834    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft"
Sep 19 01:28:51.719729 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:51.719704    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p"
Sep 19 01:28:51.771505 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:51.771458    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hnwf\" (UniqueName: \"kubernetes.io/projected/57685deb-7ca4-4c80-b48c-7b34ba3c9a8a-kube-api-access-4hnwf\") pod \"57685deb-7ca4-4c80-b48c-7b34ba3c9a8a\" (UID: \"57685deb-7ca4-4c80-b48c-7b34ba3c9a8a\") "
Sep 19 01:28:51.771672 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:51.771612    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57685deb-7ca4-4c80-b48c-7b34ba3c9a8a-plugins-config-volume\") pod \"57685deb-7ca4-4c80-b48c-7b34ba3c9a8a\" (UID: \"57685deb-7ca4-4c80-b48c-7b34ba3c9a8a\") "
Sep 19 01:28:51.771991 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:51.771967    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57685deb-7ca4-4c80-b48c-7b34ba3c9a8a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "57685deb-7ca4-4c80-b48c-7b34ba3c9a8a" (UID: "57685deb-7ca4-4c80-b48c-7b34ba3c9a8a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:28:51.773621 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:51.773597    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57685deb-7ca4-4c80-b48c-7b34ba3c9a8a-kube-api-access-4hnwf" (OuterVolumeSpecName: "kube-api-access-4hnwf") pod "57685deb-7ca4-4c80-b48c-7b34ba3c9a8a" (UID: "57685deb-7ca4-4c80-b48c-7b34ba3c9a8a"). InnerVolumeSpecName "kube-api-access-4hnwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:28:51.873146 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:51.873108    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e022024c-87f7-4283-b317-232d61243405-plugins-config-volume\") pod \"e022024c-87f7-4283-b317-232d61243405\" (UID: \"e022024c-87f7-4283-b317-232d61243405\") "
Sep 19 01:28:51.873325 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:51.873240    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w85xx\" (UniqueName: \"kubernetes.io/projected/e022024c-87f7-4283-b317-232d61243405-kube-api-access-w85xx\") pod \"e022024c-87f7-4283-b317-232d61243405\" (UID: \"e022024c-87f7-4283-b317-232d61243405\") "
Sep 19 01:28:51.873492 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:51.873469    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e022024c-87f7-4283-b317-232d61243405-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e022024c-87f7-4283-b317-232d61243405" (UID: "e022024c-87f7-4283-b317-232d61243405"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:28:51.873646 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:51.873629    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hnwf\" (UniqueName: \"kubernetes.io/projected/57685deb-7ca4-4c80-b48c-7b34ba3c9a8a-kube-api-access-4hnwf\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:28:51.873699 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:51.873647    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57685deb-7ca4-4c80-b48c-7b34ba3c9a8a-plugins-config-volume\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:28:51.873699 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:51.873657    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e022024c-87f7-4283-b317-232d61243405-plugins-config-volume\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:28:51.875329 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:51.875303    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e022024c-87f7-4283-b317-232d61243405-kube-api-access-w85xx" (OuterVolumeSpecName: "kube-api-access-w85xx") pod "e022024c-87f7-4283-b317-232d61243405" (UID: "e022024c-87f7-4283-b317-232d61243405"). InnerVolumeSpecName "kube-api-access-w85xx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:28:51.975034 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:51.974950    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w85xx\" (UniqueName: \"kubernetes.io/projected/e022024c-87f7-4283-b317-232d61243405-kube-api-access-w85xx\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:28:52.125915 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:52.125878    2580 generic.go:356] "Generic (PLEG): container finished" podID="e022024c-87f7-4283-b317-232d61243405" containerID="4a8acb7691174b45c2265f3ac82a4896bca4e609669aeb99abf079be0c1a037b" exitCode=0
Sep 19 01:28:52.126109 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:52.125945    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p"
Sep 19 01:28:52.126109 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:52.126058    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p" event={"ID":"e022024c-87f7-4283-b317-232d61243405","Type":"ContainerDied","Data":"4a8acb7691174b45c2265f3ac82a4896bca4e609669aeb99abf079be0c1a037b"}
Sep 19 01:28:52.126109 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:52.126105    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p" event={"ID":"e022024c-87f7-4283-b317-232d61243405","Type":"ContainerDied","Data":"6c7aea40141fe98da4226710bdbc4f168c7c3ba6f8d3d02f6350de646ef3be5f"}
Sep 19 01:28:52.126284 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:52.126134    2580 scope.go:117] "RemoveContainer" containerID="4a8acb7691174b45c2265f3ac82a4896bca4e609669aeb99abf079be0c1a037b"
Sep 19 01:28:52.128552 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:52.128523    2580 generic.go:356] "Generic (PLEG): container finished" podID="57685deb-7ca4-4c80-b48c-7b34ba3c9a8a" containerID="0705613923ac6ea1bfc2fdb1815204699cb8ffa34b3c0883d39e4728ad657333" exitCode=0
Sep 19 01:28:52.128673 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:52.128573    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft" event={"ID":"57685deb-7ca4-4c80-b48c-7b34ba3c9a8a","Type":"ContainerDied","Data":"0705613923ac6ea1bfc2fdb1815204699cb8ffa34b3c0883d39e4728ad657333"}
Sep 19 01:28:52.128673 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:52.128595    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft"
Sep 19 01:28:52.128673 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:52.128614    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft" event={"ID":"57685deb-7ca4-4c80-b48c-7b34ba3c9a8a","Type":"ContainerDied","Data":"cc0cfaa06a2e3069e0733574e46cf618d628cf6dd051a3a7ad8dc9fc34951683"}
Sep 19 01:28:52.139314 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:52.139276    2580 scope.go:117] "RemoveContainer" containerID="4a8acb7691174b45c2265f3ac82a4896bca4e609669aeb99abf079be0c1a037b"
Sep 19 01:28:52.139674 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:28:52.139655    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a8acb7691174b45c2265f3ac82a4896bca4e609669aeb99abf079be0c1a037b\": container with ID starting with 4a8acb7691174b45c2265f3ac82a4896bca4e609669aeb99abf079be0c1a037b not found: ID does not exist" containerID="4a8acb7691174b45c2265f3ac82a4896bca4e609669aeb99abf079be0c1a037b"
Sep 19 01:28:52.139771 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:52.139681    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a8acb7691174b45c2265f3ac82a4896bca4e609669aeb99abf079be0c1a037b"} err="failed to get container status \"4a8acb7691174b45c2265f3ac82a4896bca4e609669aeb99abf079be0c1a037b\": rpc error: code = NotFound desc = could not find container \"4a8acb7691174b45c2265f3ac82a4896bca4e609669aeb99abf079be0c1a037b\": container with ID starting with 4a8acb7691174b45c2265f3ac82a4896bca4e609669aeb99abf079be0c1a037b not found: ID does not exist"
Sep 19 01:28:52.139771 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:52.139699    2580 scope.go:117] "RemoveContainer" containerID="0705613923ac6ea1bfc2fdb1815204699cb8ffa34b3c0883d39e4728ad657333"
Sep 19 01:28:52.148651 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:52.148632    2580 scope.go:117] "RemoveContainer" containerID="0705613923ac6ea1bfc2fdb1815204699cb8ffa34b3c0883d39e4728ad657333"
Sep 19 01:28:52.148916 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:28:52.148899    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0705613923ac6ea1bfc2fdb1815204699cb8ffa34b3c0883d39e4728ad657333\": container with ID starting with 0705613923ac6ea1bfc2fdb1815204699cb8ffa34b3c0883d39e4728ad657333 not found: ID does not exist" containerID="0705613923ac6ea1bfc2fdb1815204699cb8ffa34b3c0883d39e4728ad657333"
Sep 19 01:28:52.148986 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:52.148921    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0705613923ac6ea1bfc2fdb1815204699cb8ffa34b3c0883d39e4728ad657333"} err="failed to get container status \"0705613923ac6ea1bfc2fdb1815204699cb8ffa34b3c0883d39e4728ad657333\": rpc error: code = NotFound desc = could not find container \"0705613923ac6ea1bfc2fdb1815204699cb8ffa34b3c0883d39e4728ad657333\": container with ID starting with 0705613923ac6ea1bfc2fdb1815204699cb8ffa34b3c0883d39e4728ad657333 not found: ID does not exist"
Sep 19 01:28:52.177379 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:52.177348    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p"]
Sep 19 01:28:52.191473 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:52.191451    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-b3b22c-5b84b84bbb-cfb5p"]
Sep 19 01:28:52.219755 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:52.219728    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft"]
Sep 19 01:28:52.228660 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:52.228592    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b3b22c-5d6c5d7999-49rft"]
Sep 19 01:28:53.064777 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:53.064746    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57685deb-7ca4-4c80-b48c-7b34ba3c9a8a" path="/var/lib/kubelet/pods/57685deb-7ca4-4c80-b48c-7b34ba3c9a8a/volumes"
Sep 19 01:28:53.065188 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:28:53.065171    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e022024c-87f7-4283-b317-232d61243405" path="/var/lib/kubelet/pods/e022024c-87f7-4283-b317-232d61243405/volumes"
Sep 19 01:29:27.147781 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.147747    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj"]
Sep 19 01:29:27.148400 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.148383    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7638211-a6fe-4e9a-ab6e-8bc21e4111a1" containerName="payload-pre-processing"
Sep 19 01:29:27.148449 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.148401    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7638211-a6fe-4e9a-ab6e-8bc21e4111a1" containerName="payload-pre-processing"
Sep 19 01:29:27.148449 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.148430    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e022024c-87f7-4283-b317-232d61243405" containerName="payload-pre-processing"
Sep 19 01:29:27.148449 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.148436    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e022024c-87f7-4283-b317-232d61243405" containerName="payload-pre-processing"
Sep 19 01:29:27.148449 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.148447    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57685deb-7ca4-4c80-b48c-7b34ba3c9a8a" containerName="payload-processing"
Sep 19 01:29:27.148608 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.148453    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="57685deb-7ca4-4c80-b48c-7b34ba3c9a8a" containerName="payload-processing"
Sep 19 01:29:27.148608 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.148460    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24ca7f86-c58b-4996-ab2d-ef94aef5ef92" containerName="payload-processing"
Sep 19 01:29:27.148608 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.148464    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="24ca7f86-c58b-4996-ab2d-ef94aef5ef92" containerName="payload-processing"
Sep 19 01:29:27.148608 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.148573    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7638211-a6fe-4e9a-ab6e-8bc21e4111a1" containerName="payload-pre-processing"
Sep 19 01:29:27.148608 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.148584    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="24ca7f86-c58b-4996-ab2d-ef94aef5ef92" containerName="payload-processing"
Sep 19 01:29:27.148608 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.148591    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e022024c-87f7-4283-b317-232d61243405" containerName="payload-pre-processing"
Sep 19 01:29:27.148608 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.148598    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="57685deb-7ca4-4c80-b48c-7b34ba3c9a8a" containerName="payload-processing"
Sep 19 01:29:27.152066 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.152046    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj"
Sep 19 01:29:27.154395 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.154372    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-cf0ace-dockercfg-vh2xg\""
Sep 19 01:29:27.154525 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.154410    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-cf0ace\""
Sep 19 01:29:27.164981 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.164954    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj"]
Sep 19 01:29:27.217643 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.217619    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10b03f16-117d-422a-9035-884c0b0d9b5e-plugins-config-volume\") pod \"payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj\" (UID: \"10b03f16-117d-422a-9035-884c0b0d9b5e\") " pod="openshift-ingress/payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj"
Sep 19 01:29:27.217753 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.217664    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzzj8\" (UniqueName: \"kubernetes.io/projected/10b03f16-117d-422a-9035-884c0b0d9b5e-kube-api-access-fzzj8\") pod \"payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj\" (UID: \"10b03f16-117d-422a-9035-884c0b0d9b5e\") " pod="openshift-ingress/payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj"
Sep 19 01:29:27.319117 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.319062    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzzj8\" (UniqueName: \"kubernetes.io/projected/10b03f16-117d-422a-9035-884c0b0d9b5e-kube-api-access-fzzj8\") pod \"payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj\" (UID: \"10b03f16-117d-422a-9035-884c0b0d9b5e\") " pod="openshift-ingress/payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj"
Sep 19 01:29:27.319320 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.319183    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10b03f16-117d-422a-9035-884c0b0d9b5e-plugins-config-volume\") pod \"payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj\" (UID: \"10b03f16-117d-422a-9035-884c0b0d9b5e\") " pod="openshift-ingress/payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj"
Sep 19 01:29:27.319834 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.319812    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10b03f16-117d-422a-9035-884c0b0d9b5e-plugins-config-volume\") pod \"payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj\" (UID: \"10b03f16-117d-422a-9035-884c0b0d9b5e\") " pod="openshift-ingress/payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj"
Sep 19 01:29:27.329358 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.329328    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzzj8\" (UniqueName: \"kubernetes.io/projected/10b03f16-117d-422a-9035-884c0b0d9b5e-kube-api-access-fzzj8\") pod \"payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj\" (UID: \"10b03f16-117d-422a-9035-884c0b0d9b5e\") " pod="openshift-ingress/payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj"
Sep 19 01:29:27.464329 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.464249    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj"
Sep 19 01:29:27.744486 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:27.744460    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj"]
Sep 19 01:29:27.746246 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:29:27.746220    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10b03f16_117d_422a_9035_884c0b0d9b5e.slice/crio-d71bcc1e57aa7a22109047c172ed4094d20a9d8d693f35583a259acb4526acf9 WatchSource:0}: Error finding container d71bcc1e57aa7a22109047c172ed4094d20a9d8d693f35583a259acb4526acf9: Status 404 returned error can't find the container with id d71bcc1e57aa7a22109047c172ed4094d20a9d8d693f35583a259acb4526acf9
Sep 19 01:29:28.364869 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:28.364824    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj" event={"ID":"10b03f16-117d-422a-9035-884c0b0d9b5e","Type":"ContainerStarted","Data":"2052698e0f62eda5c40b9388f0705673e9724202a0039210a22e22e207ea50ca"}
Sep 19 01:29:28.365247 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:28.364896    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj" event={"ID":"10b03f16-117d-422a-9035-884c0b0d9b5e","Type":"ContainerStarted","Data":"d71bcc1e57aa7a22109047c172ed4094d20a9d8d693f35583a259acb4526acf9"}
Sep 19 01:29:28.365247 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:28.364925    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj"
Sep 19 01:29:28.391338 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:28.391283    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj" podStartSLOduration=1.391262857 podStartE2EDuration="1.391262857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:29:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:29:28.387997695 +0000 UTC m=+1023.294859742" watchObservedRunningTime="2026-09-19 01:29:28.391262857 +0000 UTC m=+1023.298124910"
Sep 19 01:29:39.376458 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:29:39.376425    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj"
Sep 19 01:30:00.151131 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:00.151098    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5a005d-29829690-t8bp4"]
Sep 19 01:30:00.155075 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:00.155058    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5a005d-29829690-t8bp4"
Sep 19 01:30:00.158115 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:00.158094    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s7pvt\""
Sep 19 01:30:00.166357 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:00.166332    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5a005d-29829690-t8bp4"]
Sep 19 01:30:00.166473 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:00.166363    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5a005d-29829690-t8bp4"
Sep 19 01:30:00.441605 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:00.441575    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5a005d-29829690-t8bp4"]
Sep 19 01:30:00.443101 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:30:00.443072    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b2c5a78_c92d_4a03_ac8a_594a80841ac1.slice/crio-19e7cb151d0745605887ed54b8717d4093a3ae45b6289e0bd04791aaf513a5fe WatchSource:0}: Error finding container 19e7cb151d0745605887ed54b8717d4093a3ae45b6289e0bd04791aaf513a5fe: Status 404 returned error can't find the container with id 19e7cb151d0745605887ed54b8717d4093a3ae45b6289e0bd04791aaf513a5fe
Sep 19 01:30:00.445018 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:00.445002    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 01:30:00.577491 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:00.577461    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5a005d-29829690-t8bp4" event={"ID":"3b2c5a78-c92d-4a03-ac8a-594a80841ac1","Type":"ContainerStarted","Data":"19e7cb151d0745605887ed54b8717d4093a3ae45b6289e0bd04791aaf513a5fe"}
Sep 19 01:30:04.605254 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:04.605161    2580 generic.go:356] "Generic (PLEG): container finished" podID="3b2c5a78-c92d-4a03-ac8a-594a80841ac1" containerID="a64ef06986125f46e6fc75a82e4731a7d2e60aa834556006b4cd645c99a1ab28" exitCode=0
Sep 19 01:30:04.605254 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:04.605230    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5a005d-29829690-t8bp4" event={"ID":"3b2c5a78-c92d-4a03-ac8a-594a80841ac1","Type":"ContainerDied","Data":"a64ef06986125f46e6fc75a82e4731a7d2e60aa834556006b4cd645c99a1ab28"}
Sep 19 01:30:05.755686 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:05.755666    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5a005d-29829690-t8bp4"
Sep 19 01:30:06.618056 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:06.618027    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5a005d-29829690-t8bp4"
Sep 19 01:30:06.618246 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:06.618073    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5a005d-29829690-t8bp4" event={"ID":"3b2c5a78-c92d-4a03-ac8a-594a80841ac1","Type":"ContainerDied","Data":"19e7cb151d0745605887ed54b8717d4093a3ae45b6289e0bd04791aaf513a5fe"}
Sep 19 01:30:06.618246 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:06.618127    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="19e7cb151d0745605887ed54b8717d4093a3ae45b6289e0bd04791aaf513a5fe"
Sep 19 01:30:11.516589 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:11.516513    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8"]
Sep 19 01:30:11.516997 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:11.516984    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b2c5a78-c92d-4a03-ac8a-594a80841ac1" containerName="cleanup"
Sep 19 01:30:11.517044 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:11.516997    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b2c5a78-c92d-4a03-ac8a-594a80841ac1" containerName="cleanup"
Sep 19 01:30:11.517098 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:11.517090    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b2c5a78-c92d-4a03-ac8a-594a80841ac1" containerName="cleanup"
Sep 19 01:30:11.520505 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:11.520479    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8"
Sep 19 01:30:11.522699 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:11.522672    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-728f63f6\""
Sep 19 01:30:11.522831 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:11.522766    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-728f63f6-dockercfg-cj9f5\""
Sep 19 01:30:11.531514 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:11.531477    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8"]
Sep 19 01:30:11.650491 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:11.650459    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbefdc3c-9dd3-498d-b55a-2d7c4af26066-plugins-config-volume\") pod \"payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8\" (UID: \"bbefdc3c-9dd3-498d-b55a-2d7c4af26066\") " pod="openshift-ingress/payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8"
Sep 19 01:30:11.650698 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:11.650540    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhwqm\" (UniqueName: \"kubernetes.io/projected/bbefdc3c-9dd3-498d-b55a-2d7c4af26066-kube-api-access-jhwqm\") pod \"payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8\" (UID: \"bbefdc3c-9dd3-498d-b55a-2d7c4af26066\") " pod="openshift-ingress/payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8"
Sep 19 01:30:11.751513 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:11.751472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbefdc3c-9dd3-498d-b55a-2d7c4af26066-plugins-config-volume\") pod \"payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8\" (UID: \"bbefdc3c-9dd3-498d-b55a-2d7c4af26066\") " pod="openshift-ingress/payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8"
Sep 19 01:30:11.751682 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:11.751557    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhwqm\" (UniqueName: \"kubernetes.io/projected/bbefdc3c-9dd3-498d-b55a-2d7c4af26066-kube-api-access-jhwqm\") pod \"payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8\" (UID: \"bbefdc3c-9dd3-498d-b55a-2d7c4af26066\") " pod="openshift-ingress/payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8"
Sep 19 01:30:11.752256 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:11.752235    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbefdc3c-9dd3-498d-b55a-2d7c4af26066-plugins-config-volume\") pod \"payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8\" (UID: \"bbefdc3c-9dd3-498d-b55a-2d7c4af26066\") " pod="openshift-ingress/payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8"
Sep 19 01:30:11.759139 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:11.759116    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhwqm\" (UniqueName: \"kubernetes.io/projected/bbefdc3c-9dd3-498d-b55a-2d7c4af26066-kube-api-access-jhwqm\") pod \"payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8\" (UID: \"bbefdc3c-9dd3-498d-b55a-2d7c4af26066\") " pod="openshift-ingress/payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8"
Sep 19 01:30:11.834898 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:11.834823    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8"
Sep 19 01:30:12.110117 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:12.110088    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8"]
Sep 19 01:30:12.111026 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:30:12.110995    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbefdc3c_9dd3_498d_b55a_2d7c4af26066.slice/crio-5588d2258ef8c5f29644843d50fe9ff5bc9cc6638ab5a11f8f5ff51029fd540b WatchSource:0}: Error finding container 5588d2258ef8c5f29644843d50fe9ff5bc9cc6638ab5a11f8f5ff51029fd540b: Status 404 returned error can't find the container with id 5588d2258ef8c5f29644843d50fe9ff5bc9cc6638ab5a11f8f5ff51029fd540b
Sep 19 01:30:12.661720 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:12.661683    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8" event={"ID":"bbefdc3c-9dd3-498d-b55a-2d7c4af26066","Type":"ContainerStarted","Data":"38f42751b1e42155faa05ce81cc0eb2fd9a7c827b07e1c93badcde6f09d7101b"}
Sep 19 01:30:12.662091 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:12.661750    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8"
Sep 19 01:30:12.662091 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:12.661778    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8" event={"ID":"bbefdc3c-9dd3-498d-b55a-2d7c4af26066","Type":"ContainerStarted","Data":"5588d2258ef8c5f29644843d50fe9ff5bc9cc6638ab5a11f8f5ff51029fd540b"}
Sep 19 01:30:12.679752 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:12.679706    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8" podStartSLOduration=1.6796866719999999 podStartE2EDuration="1.679686672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:30:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:30:12.678917492 +0000 UTC m=+1067.585779543" watchObservedRunningTime="2026-09-19 01:30:12.679686672 +0000 UTC m=+1067.586548720"
Sep 19 01:30:23.670143 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:23.670115    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8"
Sep 19 01:30:47.964820 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:47.964789    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj"]
Sep 19 01:30:47.967894 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:47.965014    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj" podUID="10b03f16-117d-422a-9035-884c0b0d9b5e" containerName="payload-processing" containerID="cri-o://2052698e0f62eda5c40b9388f0705673e9724202a0039210a22e22e207ea50ca" gracePeriod=30
Sep 19 01:30:48.227614 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:48.227553    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj"
Sep 19 01:30:48.328757 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:48.328723    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10b03f16-117d-422a-9035-884c0b0d9b5e-plugins-config-volume\") pod \"10b03f16-117d-422a-9035-884c0b0d9b5e\" (UID: \"10b03f16-117d-422a-9035-884c0b0d9b5e\") "
Sep 19 01:30:48.328947 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:48.328838    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzzj8\" (UniqueName: \"kubernetes.io/projected/10b03f16-117d-422a-9035-884c0b0d9b5e-kube-api-access-fzzj8\") pod \"10b03f16-117d-422a-9035-884c0b0d9b5e\" (UID: \"10b03f16-117d-422a-9035-884c0b0d9b5e\") "
Sep 19 01:30:48.329194 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:48.329164    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10b03f16-117d-422a-9035-884c0b0d9b5e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "10b03f16-117d-422a-9035-884c0b0d9b5e" (UID: "10b03f16-117d-422a-9035-884c0b0d9b5e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:30:48.331047 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:48.331023    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10b03f16-117d-422a-9035-884c0b0d9b5e-kube-api-access-fzzj8" (OuterVolumeSpecName: "kube-api-access-fzzj8") pod "10b03f16-117d-422a-9035-884c0b0d9b5e" (UID: "10b03f16-117d-422a-9035-884c0b0d9b5e"). InnerVolumeSpecName "kube-api-access-fzzj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:30:48.430150 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:48.430106    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10b03f16-117d-422a-9035-884c0b0d9b5e-plugins-config-volume\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:30:48.430150 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:48.430150    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzzj8\" (UniqueName: \"kubernetes.io/projected/10b03f16-117d-422a-9035-884c0b0d9b5e-kube-api-access-fzzj8\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:30:48.913435 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:48.913398    2580 generic.go:356] "Generic (PLEG): container finished" podID="10b03f16-117d-422a-9035-884c0b0d9b5e" containerID="2052698e0f62eda5c40b9388f0705673e9724202a0039210a22e22e207ea50ca" exitCode=0
Sep 19 01:30:48.913645 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:48.913440    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj"
Sep 19 01:30:48.913645 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:48.913484    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj" event={"ID":"10b03f16-117d-422a-9035-884c0b0d9b5e","Type":"ContainerDied","Data":"2052698e0f62eda5c40b9388f0705673e9724202a0039210a22e22e207ea50ca"}
Sep 19 01:30:48.913645 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:48.913563    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj" event={"ID":"10b03f16-117d-422a-9035-884c0b0d9b5e","Type":"ContainerDied","Data":"d71bcc1e57aa7a22109047c172ed4094d20a9d8d693f35583a259acb4526acf9"}
Sep 19 01:30:48.913645 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:48.913589    2580 scope.go:117] "RemoveContainer" containerID="2052698e0f62eda5c40b9388f0705673e9724202a0039210a22e22e207ea50ca"
Sep 19 01:30:48.928040 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:48.927998    2580 scope.go:117] "RemoveContainer" containerID="2052698e0f62eda5c40b9388f0705673e9724202a0039210a22e22e207ea50ca"
Sep 19 01:30:48.928375 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:30:48.928351    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2052698e0f62eda5c40b9388f0705673e9724202a0039210a22e22e207ea50ca\": container with ID starting with 2052698e0f62eda5c40b9388f0705673e9724202a0039210a22e22e207ea50ca not found: ID does not exist" containerID="2052698e0f62eda5c40b9388f0705673e9724202a0039210a22e22e207ea50ca"
Sep 19 01:30:48.928519 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:48.928424    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2052698e0f62eda5c40b9388f0705673e9724202a0039210a22e22e207ea50ca"} err="failed to get container status \"2052698e0f62eda5c40b9388f0705673e9724202a0039210a22e22e207ea50ca\": rpc error: code = NotFound desc = could not find container \"2052698e0f62eda5c40b9388f0705673e9724202a0039210a22e22e207ea50ca\": container with ID starting with 2052698e0f62eda5c40b9388f0705673e9724202a0039210a22e22e207ea50ca not found: ID does not exist"
Sep 19 01:30:48.944004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:48.943979    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj"]
Sep 19 01:30:48.951242 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:48.951221    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-cf0ace-7885bcc98f-9tkgj"]
Sep 19 01:30:49.064649 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:49.064616    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10b03f16-117d-422a-9035-884c0b0d9b5e" path="/var/lib/kubelet/pods/10b03f16-117d-422a-9035-884c0b0d9b5e/volumes"
Sep 19 01:30:57.284085 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:57.284037    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8"]
Sep 19 01:30:57.284584 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:57.284306    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8" podUID="bbefdc3c-9dd3-498d-b55a-2d7c4af26066" containerName="payload-processing" containerID="cri-o://38f42751b1e42155faa05ce81cc0eb2fd9a7c827b07e1c93badcde6f09d7101b" gracePeriod=30
Sep 19 01:30:57.565982 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:57.565655    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8"
Sep 19 01:30:57.725891 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:57.725856    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhwqm\" (UniqueName: \"kubernetes.io/projected/bbefdc3c-9dd3-498d-b55a-2d7c4af26066-kube-api-access-jhwqm\") pod \"bbefdc3c-9dd3-498d-b55a-2d7c4af26066\" (UID: \"bbefdc3c-9dd3-498d-b55a-2d7c4af26066\") "
Sep 19 01:30:57.726106 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:57.725948    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbefdc3c-9dd3-498d-b55a-2d7c4af26066-plugins-config-volume\") pod \"bbefdc3c-9dd3-498d-b55a-2d7c4af26066\" (UID: \"bbefdc3c-9dd3-498d-b55a-2d7c4af26066\") "
Sep 19 01:30:57.726636 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:57.726596    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbefdc3c-9dd3-498d-b55a-2d7c4af26066-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bbefdc3c-9dd3-498d-b55a-2d7c4af26066" (UID: "bbefdc3c-9dd3-498d-b55a-2d7c4af26066"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:30:57.728943 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:57.728912    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbefdc3c-9dd3-498d-b55a-2d7c4af26066-kube-api-access-jhwqm" (OuterVolumeSpecName: "kube-api-access-jhwqm") pod "bbefdc3c-9dd3-498d-b55a-2d7c4af26066" (UID: "bbefdc3c-9dd3-498d-b55a-2d7c4af26066"). InnerVolumeSpecName "kube-api-access-jhwqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:30:57.827136 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:57.827057    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhwqm\" (UniqueName: \"kubernetes.io/projected/bbefdc3c-9dd3-498d-b55a-2d7c4af26066-kube-api-access-jhwqm\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:30:57.827136 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:57.827084    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbefdc3c-9dd3-498d-b55a-2d7c4af26066-plugins-config-volume\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:30:57.974569 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:57.974536    2580 generic.go:356] "Generic (PLEG): container finished" podID="bbefdc3c-9dd3-498d-b55a-2d7c4af26066" containerID="38f42751b1e42155faa05ce81cc0eb2fd9a7c827b07e1c93badcde6f09d7101b" exitCode=0
Sep 19 01:30:57.974757 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:57.974579    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8" event={"ID":"bbefdc3c-9dd3-498d-b55a-2d7c4af26066","Type":"ContainerDied","Data":"38f42751b1e42155faa05ce81cc0eb2fd9a7c827b07e1c93badcde6f09d7101b"}
Sep 19 01:30:57.974757 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:57.974619    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8"
Sep 19 01:30:57.974757 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:57.974634    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8" event={"ID":"bbefdc3c-9dd3-498d-b55a-2d7c4af26066","Type":"ContainerDied","Data":"5588d2258ef8c5f29644843d50fe9ff5bc9cc6638ab5a11f8f5ff51029fd540b"}
Sep 19 01:30:57.974757 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:57.974663    2580 scope.go:117] "RemoveContainer" containerID="38f42751b1e42155faa05ce81cc0eb2fd9a7c827b07e1c93badcde6f09d7101b"
Sep 19 01:30:57.985565 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:57.985548    2580 scope.go:117] "RemoveContainer" containerID="38f42751b1e42155faa05ce81cc0eb2fd9a7c827b07e1c93badcde6f09d7101b"
Sep 19 01:30:57.985868 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:30:57.985850    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38f42751b1e42155faa05ce81cc0eb2fd9a7c827b07e1c93badcde6f09d7101b\": container with ID starting with 38f42751b1e42155faa05ce81cc0eb2fd9a7c827b07e1c93badcde6f09d7101b not found: ID does not exist" containerID="38f42751b1e42155faa05ce81cc0eb2fd9a7c827b07e1c93badcde6f09d7101b"
Sep 19 01:30:57.985912 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:57.985876    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38f42751b1e42155faa05ce81cc0eb2fd9a7c827b07e1c93badcde6f09d7101b"} err="failed to get container status \"38f42751b1e42155faa05ce81cc0eb2fd9a7c827b07e1c93badcde6f09d7101b\": rpc error: code = NotFound desc = could not find container \"38f42751b1e42155faa05ce81cc0eb2fd9a7c827b07e1c93badcde6f09d7101b\": container with ID starting with 38f42751b1e42155faa05ce81cc0eb2fd9a7c827b07e1c93badcde6f09d7101b not found: ID does not exist"
Sep 19 01:30:58.002238 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:58.002212    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8"]
Sep 19 01:30:58.011583 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:58.011563    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-728f63f6-657bc55cf7-2kpd8"]
Sep 19 01:30:59.063903 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:30:59.063871    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbefdc3c-9dd3-498d-b55a-2d7c4af26066" path="/var/lib/kubelet/pods/bbefdc3c-9dd3-498d-b55a-2d7c4af26066/volumes"
Sep 19 01:31:24.942438 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:24.942394    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx"]
Sep 19 01:31:24.943205 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:24.943186    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10b03f16-117d-422a-9035-884c0b0d9b5e" containerName="payload-processing"
Sep 19 01:31:24.943257 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:24.943210    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="10b03f16-117d-422a-9035-884c0b0d9b5e" containerName="payload-processing"
Sep 19 01:31:24.943296 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:24.943264    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbefdc3c-9dd3-498d-b55a-2d7c4af26066" containerName="payload-processing"
Sep 19 01:31:24.943296 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:24.943272    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbefdc3c-9dd3-498d-b55a-2d7c4af26066" containerName="payload-processing"
Sep 19 01:31:24.943396 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:24.943382    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbefdc3c-9dd3-498d-b55a-2d7c4af26066" containerName="payload-processing"
Sep 19 01:31:24.943432 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:24.943414    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="10b03f16-117d-422a-9035-884c0b0d9b5e" containerName="payload-processing"
Sep 19 01:31:24.947608 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:24.947587    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx"
Sep 19 01:31:24.949921 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:24.949898    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-10325b-serving-cert\""
Sep 19 01:31:24.960962 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:24.960859    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx"]
Sep 19 01:31:25.097432 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.097390    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsp2s\" (UniqueName: \"kubernetes.io/projected/1c0d4ade-2a79-4833-8c42-306dd75927c6-kube-api-access-vsp2s\") pod \"maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx\" (UID: \"1c0d4ade-2a79-4833-8c42-306dd75927c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx"
Sep 19 01:31:25.097619 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.097539    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c0d4ade-2a79-4833-8c42-306dd75927c6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx\" (UID: \"1c0d4ade-2a79-4833-8c42-306dd75927c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx"
Sep 19 01:31:25.097619 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.097584    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c0d4ade-2a79-4833-8c42-306dd75927c6-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx\" (UID: \"1c0d4ade-2a79-4833-8c42-306dd75927c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx"
Sep 19 01:31:25.097619 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.097605    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c0d4ade-2a79-4833-8c42-306dd75927c6-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx\" (UID: \"1c0d4ade-2a79-4833-8c42-306dd75927c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx"
Sep 19 01:31:25.198535 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.198437    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c0d4ade-2a79-4833-8c42-306dd75927c6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx\" (UID: \"1c0d4ade-2a79-4833-8c42-306dd75927c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx"
Sep 19 01:31:25.198535 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.198490    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c0d4ade-2a79-4833-8c42-306dd75927c6-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx\" (UID: \"1c0d4ade-2a79-4833-8c42-306dd75927c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx"
Sep 19 01:31:25.198718 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.198540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c0d4ade-2a79-4833-8c42-306dd75927c6-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx\" (UID: \"1c0d4ade-2a79-4833-8c42-306dd75927c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx"
Sep 19 01:31:25.198718 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.198577    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsp2s\" (UniqueName: \"kubernetes.io/projected/1c0d4ade-2a79-4833-8c42-306dd75927c6-kube-api-access-vsp2s\") pod \"maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx\" (UID: \"1c0d4ade-2a79-4833-8c42-306dd75927c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx"
Sep 19 01:31:25.199242 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.199218    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c0d4ade-2a79-4833-8c42-306dd75927c6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx\" (UID: \"1c0d4ade-2a79-4833-8c42-306dd75927c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx"
Sep 19 01:31:25.201108 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.201084    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c0d4ade-2a79-4833-8c42-306dd75927c6-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx\" (UID: \"1c0d4ade-2a79-4833-8c42-306dd75927c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx"
Sep 19 01:31:25.201193 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.201126    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c0d4ade-2a79-4833-8c42-306dd75927c6-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx\" (UID: \"1c0d4ade-2a79-4833-8c42-306dd75927c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx"
Sep 19 01:31:25.210165 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.210130    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsp2s\" (UniqueName: \"kubernetes.io/projected/1c0d4ade-2a79-4833-8c42-306dd75927c6-kube-api-access-vsp2s\") pod \"maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx\" (UID: \"1c0d4ade-2a79-4833-8c42-306dd75927c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx"
Sep 19 01:31:25.260181 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.260146    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx"
Sep 19 01:31:25.539636 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.539610    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx"]
Sep 19 01:31:25.541434 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:31:25.541402    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c0d4ade_2a79_4833_8c42_306dd75927c6.slice/crio-7703d23fe0d24ecb85e2c8069b52a53111b28033f80b819d96d2fcdb62510bbc WatchSource:0}: Error finding container 7703d23fe0d24ecb85e2c8069b52a53111b28033f80b819d96d2fcdb62510bbc: Status 404 returned error can't find the container with id 7703d23fe0d24ecb85e2c8069b52a53111b28033f80b819d96d2fcdb62510bbc
Sep 19 01:31:25.671414 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.671363    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj"]
Sep 19 01:31:25.677309 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.677288    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj"
Sep 19 01:31:25.679755 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.679739    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-10325b\""
Sep 19 01:31:25.680887 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.680861    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-10325b-dockercfg-v64fc\""
Sep 19 01:31:25.689237 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.689203    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj"]
Sep 19 01:31:25.806698 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.806596    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca421c49-d4f8-477c-8974-70ffc05b9468-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj\" (UID: \"ca421c49-d4f8-477c-8974-70ffc05b9468\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj"
Sep 19 01:31:25.806698 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.806645    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvrj4\" (UniqueName: \"kubernetes.io/projected/ca421c49-d4f8-477c-8974-70ffc05b9468-kube-api-access-cvrj4\") pod \"payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj\" (UID: \"ca421c49-d4f8-477c-8974-70ffc05b9468\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj"
Sep 19 01:31:25.907687 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.907652    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca421c49-d4f8-477c-8974-70ffc05b9468-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj\" (UID: \"ca421c49-d4f8-477c-8974-70ffc05b9468\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj"
Sep 19 01:31:25.907838 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.907698    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvrj4\" (UniqueName: \"kubernetes.io/projected/ca421c49-d4f8-477c-8974-70ffc05b9468-kube-api-access-cvrj4\") pod \"payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj\" (UID: \"ca421c49-d4f8-477c-8974-70ffc05b9468\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj"
Sep 19 01:31:25.908392 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.908370    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca421c49-d4f8-477c-8974-70ffc05b9468-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj\" (UID: \"ca421c49-d4f8-477c-8974-70ffc05b9468\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj"
Sep 19 01:31:25.916273 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.916254    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvrj4\" (UniqueName: \"kubernetes.io/projected/ca421c49-d4f8-477c-8974-70ffc05b9468-kube-api-access-cvrj4\") pod \"payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj\" (UID: \"ca421c49-d4f8-477c-8974-70ffc05b9468\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj"
Sep 19 01:31:25.990616 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:25.990593    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj"
Sep 19 01:31:26.157006 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:26.156913    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx" event={"ID":"1c0d4ade-2a79-4833-8c42-306dd75927c6","Type":"ContainerStarted","Data":"15e2c0e43e6f08822f86c48947444516ee1f7940097b929520c1e6ad2d46e4a9"}
Sep 19 01:31:26.157006 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:26.156974    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx" event={"ID":"1c0d4ade-2a79-4833-8c42-306dd75927c6","Type":"ContainerStarted","Data":"7703d23fe0d24ecb85e2c8069b52a53111b28033f80b819d96d2fcdb62510bbc"}
Sep 19 01:31:26.157223 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:26.157115    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx"
Sep 19 01:31:26.175651 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:26.175606    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx" podStartSLOduration=1.808282521 podStartE2EDuration="2.175591633s" totalImagesPullingTime="367.309112ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:31:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:31:26.173646304 +0000 UTC m=+1141.080508351" watchObservedRunningTime="2026-09-19 01:31:26.175591633 +0000 UTC m=+1141.082453680"
Sep 19 01:31:26.272706 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:26.272674    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj"]
Sep 19 01:31:26.273729 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:31:26.273699    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca421c49_d4f8_477c_8974_70ffc05b9468.slice/crio-e1126e93177e0777f829a0aded8c92112611faa37d1c6713c402ed008ec8c009 WatchSource:0}: Error finding container e1126e93177e0777f829a0aded8c92112611faa37d1c6713c402ed008ec8c009: Status 404 returned error can't find the container with id e1126e93177e0777f829a0aded8c92112611faa37d1c6713c402ed008ec8c009
Sep 19 01:31:27.165935 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:27.165894    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj" event={"ID":"ca421c49-d4f8-477c-8974-70ffc05b9468","Type":"ContainerStarted","Data":"bd62a794981dee7fc6876b0638a87ff063efe73bd026f7505bf02da13e9edc74"}
Sep 19 01:31:27.166318 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:27.165969    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj"
Sep 19 01:31:27.166318 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:27.165994    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj" event={"ID":"ca421c49-d4f8-477c-8974-70ffc05b9468","Type":"ContainerStarted","Data":"e1126e93177e0777f829a0aded8c92112611faa37d1c6713c402ed008ec8c009"}
Sep 19 01:31:27.184814 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:27.184770    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj" podStartSLOduration=2.184752664 podStartE2EDuration="2.184752664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:31:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:31:27.183973435 +0000 UTC m=+1142.090835488" watchObservedRunningTime="2026-09-19 01:31:27.184752664 +0000 UTC m=+1142.091614723"
Sep 19 01:31:32.172338 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:32.172308    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx"
Sep 19 01:31:38.175626 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:31:38.175592    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj"
Sep 19 01:32:17.775022 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:17.774980    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7503-35b4c1"]
Sep 19 01:32:17.778832 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:17.778814    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-35b4c1"
Sep 19 01:32:17.782057 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:17.781947    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-frkf9\""
Sep 19 01:32:17.785397 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:17.785374    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7503-35b4c1"]
Sep 19 01:32:17.924838 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:17.924802    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxsmc\" (UniqueName: \"kubernetes.io/projected/ad4f996a-b04c-4939-a081-8dba1dcc2ffe-kube-api-access-lxsmc\") pod \"test-curl-7503-35b4c1\" (UID: \"ad4f996a-b04c-4939-a081-8dba1dcc2ffe\") " pod="opendatahub/test-curl-7503-35b4c1"
Sep 19 01:32:18.026671 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:18.026572    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxsmc\" (UniqueName: \"kubernetes.io/projected/ad4f996a-b04c-4939-a081-8dba1dcc2ffe-kube-api-access-lxsmc\") pod \"test-curl-7503-35b4c1\" (UID: \"ad4f996a-b04c-4939-a081-8dba1dcc2ffe\") " pod="opendatahub/test-curl-7503-35b4c1"
Sep 19 01:32:18.037457 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:18.037422    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxsmc\" (UniqueName: \"kubernetes.io/projected/ad4f996a-b04c-4939-a081-8dba1dcc2ffe-kube-api-access-lxsmc\") pod \"test-curl-7503-35b4c1\" (UID: \"ad4f996a-b04c-4939-a081-8dba1dcc2ffe\") " pod="opendatahub/test-curl-7503-35b4c1"
Sep 19 01:32:18.091258 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:18.091227    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-35b4c1"
Sep 19 01:32:18.372306 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:18.372280    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7503-35b4c1"]
Sep 19 01:32:18.374215 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:32:18.374189    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad4f996a_b04c_4939_a081_8dba1dcc2ffe.slice/crio-ab4583dfdf7d38e3dc28a0aa7807d0117f426fa8e2ab5d6bd5b18399d692c4ee WatchSource:0}: Error finding container ab4583dfdf7d38e3dc28a0aa7807d0117f426fa8e2ab5d6bd5b18399d692c4ee: Status 404 returned error can't find the container with id ab4583dfdf7d38e3dc28a0aa7807d0117f426fa8e2ab5d6bd5b18399d692c4ee
Sep 19 01:32:18.518210 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:18.518175    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7503-35b4c1" event={"ID":"ad4f996a-b04c-4939-a081-8dba1dcc2ffe","Type":"ContainerStarted","Data":"31755e02b6b035acd0ebd4c675eff1b6b39a9ac31458187e8cbd55cbe92aa460"}
Sep 19 01:32:18.518352 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:18.518219    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7503-35b4c1" event={"ID":"ad4f996a-b04c-4939-a081-8dba1dcc2ffe","Type":"ContainerStarted","Data":"ab4583dfdf7d38e3dc28a0aa7807d0117f426fa8e2ab5d6bd5b18399d692c4ee"}
Sep 19 01:32:19.298835 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:19.298731    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7503-35b4c1" podStartSLOduration=2.298713945 podStartE2EDuration="2.298713945s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:32:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:32:18.557830794 +0000 UTC m=+1193.464692843" watchObservedRunningTime="2026-09-19 01:32:19.298713945 +0000 UTC m=+1194.205576020"
Sep 19 01:32:19.299262 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:19.299247    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7503-35b4c1"]
Sep 19 01:32:19.304061 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:19.304036    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7503-35b4c1"]
Sep 19 01:32:19.462130 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:19.462099    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7503-d8d841"]
Sep 19 01:32:19.462613 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:19.462599    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad4f996a-b04c-4939-a081-8dba1dcc2ffe" containerName="test-curl-7503-35b4c1"
Sep 19 01:32:19.462663 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:19.462615    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad4f996a-b04c-4939-a081-8dba1dcc2ffe" containerName="test-curl-7503-35b4c1"
Sep 19 01:32:19.462717 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:19.462708    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad4f996a-b04c-4939-a081-8dba1dcc2ffe" containerName="test-curl-7503-35b4c1"
Sep 19 01:32:19.465996 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:19.465979    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-d8d841"
Sep 19 01:32:19.478954 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:19.478928    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7503-d8d841"]
Sep 19 01:32:19.524391 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:19.524363    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7503-35b4c1" container="test-curl-7503-35b4c1"
Sep 19 01:32:19.543001 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:19.542960    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hgbh\" (UniqueName: \"kubernetes.io/projected/89e23b58-91b7-4481-9b0c-47c949439f3d-kube-api-access-6hgbh\") pod \"test-curl-7503-d8d841\" (UID: \"89e23b58-91b7-4481-9b0c-47c949439f3d\") " pod="opendatahub/test-curl-7503-d8d841"
Sep 19 01:32:19.644014 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:19.643925    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hgbh\" (UniqueName: \"kubernetes.io/projected/89e23b58-91b7-4481-9b0c-47c949439f3d-kube-api-access-6hgbh\") pod \"test-curl-7503-d8d841\" (UID: \"89e23b58-91b7-4481-9b0c-47c949439f3d\") " pod="opendatahub/test-curl-7503-d8d841"
Sep 19 01:32:19.652996 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:19.652969    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hgbh\" (UniqueName: \"kubernetes.io/projected/89e23b58-91b7-4481-9b0c-47c949439f3d-kube-api-access-6hgbh\") pod \"test-curl-7503-d8d841\" (UID: \"89e23b58-91b7-4481-9b0c-47c949439f3d\") " pod="opendatahub/test-curl-7503-d8d841"
Sep 19 01:32:19.776374 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:19.776339    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-d8d841"
Sep 19 01:32:20.054416 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:20.054393    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7503-d8d841"]
Sep 19 01:32:20.056016 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:32:20.055987    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89e23b58_91b7_4481_9b0c_47c949439f3d.slice/crio-47e75ac5873a11673540fe094c47dc1ff584cc5647823c9997321f1f3b9016b4 WatchSource:0}: Error finding container 47e75ac5873a11673540fe094c47dc1ff584cc5647823c9997321f1f3b9016b4: Status 404 returned error can't find the container with id 47e75ac5873a11673540fe094c47dc1ff584cc5647823c9997321f1f3b9016b4
Sep 19 01:32:20.535965 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:20.535926    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7503-d8d841" event={"ID":"89e23b58-91b7-4481-9b0c-47c949439f3d","Type":"ContainerStarted","Data":"f5a763f78932f0a9dbf278c13fb9594ec965026d1703b139d5dc0bfc49184a38"}
Sep 19 01:32:20.536474 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:20.535983    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7503-d8d841" event={"ID":"89e23b58-91b7-4481-9b0c-47c949439f3d","Type":"ContainerStarted","Data":"47e75ac5873a11673540fe094c47dc1ff584cc5647823c9997321f1f3b9016b4"}
Sep 19 01:32:20.536474 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:20.536218    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7503-35b4c1" podUID="ad4f996a-b04c-4939-a081-8dba1dcc2ffe" containerName="test-curl-7503-35b4c1" containerID="cri-o://31755e02b6b035acd0ebd4c675eff1b6b39a9ac31458187e8cbd55cbe92aa460" gracePeriod=2
Sep 19 01:32:20.567976 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:20.567943    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7503-35b4c1\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="ad4f996a-b04c-4939-a081-8dba1dcc2ffe" pod="opendatahub/test-curl-7503-35b4c1"
Sep 19 01:32:20.568176 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:20.568133    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7503-d8d841" podStartSLOduration=1.5681186010000001 podStartE2EDuration="1.568118601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:32:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:32:20.566012987 +0000 UTC m=+1195.472875035" watchObservedRunningTime="2026-09-19 01:32:20.568118601 +0000 UTC m=+1195.474980699"
Sep 19 01:32:21.236442 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:21.236410    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7503-d8d841"]
Sep 19 01:32:21.240211 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:21.240142    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7503-d8d841"]
Sep 19 01:32:21.361156 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:21.361121    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7503-ab7bf8"]
Sep 19 01:32:21.361709 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:21.361695    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89e23b58-91b7-4481-9b0c-47c949439f3d" containerName="test-curl-7503-d8d841"
Sep 19 01:32:21.361766 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:21.361711    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="89e23b58-91b7-4481-9b0c-47c949439f3d" containerName="test-curl-7503-d8d841"
Sep 19 01:32:21.361819 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:21.361810    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="89e23b58-91b7-4481-9b0c-47c949439f3d" containerName="test-curl-7503-d8d841"
Sep 19 01:32:21.365337 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:21.365317    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-ab7bf8"
Sep 19 01:32:21.378548 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:21.378522    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7503-ab7bf8"]
Sep 19 01:32:21.457964 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:21.457933    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t649t\" (UniqueName: \"kubernetes.io/projected/14464d8a-681d-4b8e-8206-a4ed647e8aef-kube-api-access-t649t\") pod \"test-curl-7503-ab7bf8\" (UID: \"14464d8a-681d-4b8e-8206-a4ed647e8aef\") " pod="opendatahub/test-curl-7503-ab7bf8"
Sep 19 01:32:21.542877 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:21.542805    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7503-d8d841" container="test-curl-7503-d8d841"
Sep 19 01:32:21.558539 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:21.558515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t649t\" (UniqueName: \"kubernetes.io/projected/14464d8a-681d-4b8e-8206-a4ed647e8aef-kube-api-access-t649t\") pod \"test-curl-7503-ab7bf8\" (UID: \"14464d8a-681d-4b8e-8206-a4ed647e8aef\") " pod="opendatahub/test-curl-7503-ab7bf8"
Sep 19 01:32:21.570858 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:21.570839    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t649t\" (UniqueName: \"kubernetes.io/projected/14464d8a-681d-4b8e-8206-a4ed647e8aef-kube-api-access-t649t\") pod \"test-curl-7503-ab7bf8\" (UID: \"14464d8a-681d-4b8e-8206-a4ed647e8aef\") " pod="opendatahub/test-curl-7503-ab7bf8"
Sep 19 01:32:21.690634 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:21.690606    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-ab7bf8"
Sep 19 01:32:21.984057 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:21.984029    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7503-ab7bf8"]
Sep 19 01:32:21.986007 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:32:21.985979    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14464d8a_681d_4b8e_8206_a4ed647e8aef.slice/crio-247d84df60f101eaacf0e83409670a3eebc543350966f4a06c0e20a54d1384e6 WatchSource:0}: Error finding container 247d84df60f101eaacf0e83409670a3eebc543350966f4a06c0e20a54d1384e6: Status 404 returned error can't find the container with id 247d84df60f101eaacf0e83409670a3eebc543350966f4a06c0e20a54d1384e6
Sep 19 01:32:22.551183 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:22.551146    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7503-ab7bf8" event={"ID":"14464d8a-681d-4b8e-8206-a4ed647e8aef","Type":"ContainerStarted","Data":"53bf62dbd5270ebae51f00219eb7c9ffa9bc9357fc1b049169bd3692b4867a49"}
Sep 19 01:32:22.551457 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:22.551202    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7503-ab7bf8" event={"ID":"14464d8a-681d-4b8e-8206-a4ed647e8aef","Type":"ContainerStarted","Data":"247d84df60f101eaacf0e83409670a3eebc543350966f4a06c0e20a54d1384e6"}
Sep 19 01:32:22.551457 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:22.551398    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7503-d8d841" podUID="89e23b58-91b7-4481-9b0c-47c949439f3d" containerName="test-curl-7503-d8d841" containerID="cri-o://f5a763f78932f0a9dbf278c13fb9594ec965026d1703b139d5dc0bfc49184a38" gracePeriod=2
Sep 19 01:32:22.570925 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:22.570851    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7503-ab7bf8" podStartSLOduration=1.5708351170000001 podStartE2EDuration="1.570835117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:32:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:32:22.569917697 +0000 UTC m=+1197.476779745" watchObservedRunningTime="2026-09-19 01:32:22.570835117 +0000 UTC m=+1197.477697165"
Sep 19 01:32:22.571757 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:22.571728    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7503-d8d841\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="89e23b58-91b7-4481-9b0c-47c949439f3d" pod="opendatahub/test-curl-7503-d8d841"
Sep 19 01:32:22.736624 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:22.736582    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-35b4c1"
Sep 19 01:32:22.738854 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:22.738831    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7503-d8d841\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="89e23b58-91b7-4481-9b0c-47c949439f3d" pod="opendatahub/test-curl-7503-d8d841"
Sep 19 01:32:22.770923 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:22.770900    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxsmc\" (UniqueName: \"kubernetes.io/projected/ad4f996a-b04c-4939-a081-8dba1dcc2ffe-kube-api-access-lxsmc\") pod \"ad4f996a-b04c-4939-a081-8dba1dcc2ffe\" (UID: \"ad4f996a-b04c-4939-a081-8dba1dcc2ffe\") "
Sep 19 01:32:22.773036 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:22.773014    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad4f996a-b04c-4939-a081-8dba1dcc2ffe-kube-api-access-lxsmc" (OuterVolumeSpecName: "kube-api-access-lxsmc") pod "ad4f996a-b04c-4939-a081-8dba1dcc2ffe" (UID: "ad4f996a-b04c-4939-a081-8dba1dcc2ffe"). InnerVolumeSpecName "kube-api-access-lxsmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:32:22.872042 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:22.871976    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxsmc\" (UniqueName: \"kubernetes.io/projected/ad4f996a-b04c-4939-a081-8dba1dcc2ffe-kube-api-access-lxsmc\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:32:23.065867 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.065832    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad4f996a-b04c-4939-a081-8dba1dcc2ffe" path="/var/lib/kubelet/pods/ad4f996a-b04c-4939-a081-8dba1dcc2ffe/volumes"
Sep 19 01:32:23.208397 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.208297    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn"]
Sep 19 01:32:23.208590 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.208543    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn" podUID="29965315-e87d-4bdb-aa4b-e8bc7ed5d275" containerName="payload-processing" containerID="cri-o://e6431e0ed2f9005cb4ce28537e2cc9babb6d62405550b37a223006f1d279d0dd" gracePeriod=30
Sep 19 01:32:23.211549 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.211526    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft"]
Sep 19 01:32:23.211900 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.211871    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft" podUID="5e0b0a82-a21a-4e31-8409-09c208370b98" containerName="payload-pre-processing" containerID="cri-o://4531bc1f125ee9506fed2df303f4e1cb9088f1d00a4ed79124e871d3f427e832" gracePeriod=30
Sep 19 01:32:23.265487 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.265459    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7503-ab7bf8"]
Sep 19 01:32:23.269448 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.269414    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7503-ab7bf8"]
Sep 19 01:32:23.398631 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.398542    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7503-da9634"]
Sep 19 01:32:23.399119 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.399101    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14464d8a-681d-4b8e-8206-a4ed647e8aef" containerName="test-curl-7503-ab7bf8"
Sep 19 01:32:23.399177 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.399122    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="14464d8a-681d-4b8e-8206-a4ed647e8aef" containerName="test-curl-7503-ab7bf8"
Sep 19 01:32:23.399221 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.399207    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="14464d8a-681d-4b8e-8206-a4ed647e8aef" containerName="test-curl-7503-ab7bf8"
Sep 19 01:32:23.402659 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.402642    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-da9634"
Sep 19 01:32:23.412410 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.411567    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7503-da9634"]
Sep 19 01:32:23.477846 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.477753    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vtfs\" (UniqueName: \"kubernetes.io/projected/106fbc82-54ec-490b-9bf2-2613ca4e27ef-kube-api-access-2vtfs\") pod \"test-curl-7503-da9634\" (UID: \"106fbc82-54ec-490b-9bf2-2613ca4e27ef\") " pod="opendatahub/test-curl-7503-da9634"
Sep 19 01:32:23.493361 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.493333    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn"
Sep 19 01:32:23.506962 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.506943    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft"
Sep 19 01:32:23.559987 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.559949    2580 generic.go:356] "Generic (PLEG): container finished" podID="ad4f996a-b04c-4939-a081-8dba1dcc2ffe" containerID="31755e02b6b035acd0ebd4c675eff1b6b39a9ac31458187e8cbd55cbe92aa460" exitCode=137
Sep 19 01:32:23.560444 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.560026    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-35b4c1"
Sep 19 01:32:23.560444 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.560117    2580 scope.go:117] "RemoveContainer" containerID="31755e02b6b035acd0ebd4c675eff1b6b39a9ac31458187e8cbd55cbe92aa460"
Sep 19 01:32:23.562069 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.561932    2580 generic.go:356] "Generic (PLEG): container finished" podID="5e0b0a82-a21a-4e31-8409-09c208370b98" containerID="4531bc1f125ee9506fed2df303f4e1cb9088f1d00a4ed79124e871d3f427e832" exitCode=0
Sep 19 01:32:23.562069 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.561972    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft"
Sep 19 01:32:23.562069 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.562048    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft" event={"ID":"5e0b0a82-a21a-4e31-8409-09c208370b98","Type":"ContainerDied","Data":"4531bc1f125ee9506fed2df303f4e1cb9088f1d00a4ed79124e871d3f427e832"}
Sep 19 01:32:23.562330 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.562096    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft" event={"ID":"5e0b0a82-a21a-4e31-8409-09c208370b98","Type":"ContainerDied","Data":"7eef2f1ef09a4f526e5d01f91dcbb42e61dc8f9cb36164ef3aa6652295107cb7"}
Sep 19 01:32:23.564111 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.564088    2580 generic.go:356] "Generic (PLEG): container finished" podID="29965315-e87d-4bdb-aa4b-e8bc7ed5d275" containerID="e6431e0ed2f9005cb4ce28537e2cc9babb6d62405550b37a223006f1d279d0dd" exitCode=0
Sep 19 01:32:23.564230 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.564148    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn" event={"ID":"29965315-e87d-4bdb-aa4b-e8bc7ed5d275","Type":"ContainerDied","Data":"e6431e0ed2f9005cb4ce28537e2cc9babb6d62405550b37a223006f1d279d0dd"}
Sep 19 01:32:23.564230 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.564182    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn" event={"ID":"29965315-e87d-4bdb-aa4b-e8bc7ed5d275","Type":"ContainerDied","Data":"1d60c2289530a747f14d772da598c4e5f20f9e8063bb71403f363f2313b41598"}
Sep 19 01:32:23.564230 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.564190    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn"
Sep 19 01:32:23.564609 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:23.564590    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7503-ab7bf8" container="test-curl-7503-ab7bf8"
Sep 19 01:32:23.575721 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.575698    2580 scope.go:117] "RemoveContainer" containerID="31755e02b6b035acd0ebd4c675eff1b6b39a9ac31458187e8cbd55cbe92aa460"
Sep 19 01:32:23.577000 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:23.576980    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31755e02b6b035acd0ebd4c675eff1b6b39a9ac31458187e8cbd55cbe92aa460\": container with ID starting with 31755e02b6b035acd0ebd4c675eff1b6b39a9ac31458187e8cbd55cbe92aa460 not found: ID does not exist" containerID="31755e02b6b035acd0ebd4c675eff1b6b39a9ac31458187e8cbd55cbe92aa460"
Sep 19 01:32:23.577090 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.577007    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31755e02b6b035acd0ebd4c675eff1b6b39a9ac31458187e8cbd55cbe92aa460"} err="failed to get container status \"31755e02b6b035acd0ebd4c675eff1b6b39a9ac31458187e8cbd55cbe92aa460\": rpc error: code = NotFound desc = could not find container \"31755e02b6b035acd0ebd4c675eff1b6b39a9ac31458187e8cbd55cbe92aa460\": container with ID starting with 31755e02b6b035acd0ebd4c675eff1b6b39a9ac31458187e8cbd55cbe92aa460 not found: ID does not exist"
Sep 19 01:32:23.577090 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.577023    2580 scope.go:117] "RemoveContainer" containerID="4531bc1f125ee9506fed2df303f4e1cb9088f1d00a4ed79124e871d3f427e832"
Sep 19 01:32:23.578257 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.578236    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e0b0a82-a21a-4e31-8409-09c208370b98-plugins-config-volume\") pod \"5e0b0a82-a21a-4e31-8409-09c208370b98\" (UID: \"5e0b0a82-a21a-4e31-8409-09c208370b98\") "
Sep 19 01:32:23.578373 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.578292    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-276lz\" (UniqueName: \"kubernetes.io/projected/29965315-e87d-4bdb-aa4b-e8bc7ed5d275-kube-api-access-276lz\") pod \"29965315-e87d-4bdb-aa4b-e8bc7ed5d275\" (UID: \"29965315-e87d-4bdb-aa4b-e8bc7ed5d275\") "
Sep 19 01:32:23.578373 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.578323    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfslp\" (UniqueName: \"kubernetes.io/projected/5e0b0a82-a21a-4e31-8409-09c208370b98-kube-api-access-gfslp\") pod \"5e0b0a82-a21a-4e31-8409-09c208370b98\" (UID: \"5e0b0a82-a21a-4e31-8409-09c208370b98\") "
Sep 19 01:32:23.578491 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.578379    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29965315-e87d-4bdb-aa4b-e8bc7ed5d275-plugins-config-volume\") pod \"29965315-e87d-4bdb-aa4b-e8bc7ed5d275\" (UID: \"29965315-e87d-4bdb-aa4b-e8bc7ed5d275\") "
Sep 19 01:32:23.578614 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.578592    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vtfs\" (UniqueName: \"kubernetes.io/projected/106fbc82-54ec-490b-9bf2-2613ca4e27ef-kube-api-access-2vtfs\") pod \"test-curl-7503-da9634\" (UID: \"106fbc82-54ec-490b-9bf2-2613ca4e27ef\") " pod="opendatahub/test-curl-7503-da9634"
Sep 19 01:32:23.578808 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.578770    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e0b0a82-a21a-4e31-8409-09c208370b98-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5e0b0a82-a21a-4e31-8409-09c208370b98" (UID: "5e0b0a82-a21a-4e31-8409-09c208370b98"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:32:23.578904 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.578816    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29965315-e87d-4bdb-aa4b-e8bc7ed5d275-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "29965315-e87d-4bdb-aa4b-e8bc7ed5d275" (UID: "29965315-e87d-4bdb-aa4b-e8bc7ed5d275"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:32:23.580477 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.580455    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e0b0a82-a21a-4e31-8409-09c208370b98-kube-api-access-gfslp" (OuterVolumeSpecName: "kube-api-access-gfslp") pod "5e0b0a82-a21a-4e31-8409-09c208370b98" (UID: "5e0b0a82-a21a-4e31-8409-09c208370b98"). InnerVolumeSpecName "kube-api-access-gfslp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:32:23.580699 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.580675    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29965315-e87d-4bdb-aa4b-e8bc7ed5d275-kube-api-access-276lz" (OuterVolumeSpecName: "kube-api-access-276lz") pod "29965315-e87d-4bdb-aa4b-e8bc7ed5d275" (UID: "29965315-e87d-4bdb-aa4b-e8bc7ed5d275"). InnerVolumeSpecName "kube-api-access-276lz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:32:23.586429 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.586412    2580 scope.go:117] "RemoveContainer" containerID="4531bc1f125ee9506fed2df303f4e1cb9088f1d00a4ed79124e871d3f427e832"
Sep 19 01:32:23.586718 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:23.586697    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4531bc1f125ee9506fed2df303f4e1cb9088f1d00a4ed79124e871d3f427e832\": container with ID starting with 4531bc1f125ee9506fed2df303f4e1cb9088f1d00a4ed79124e871d3f427e832 not found: ID does not exist" containerID="4531bc1f125ee9506fed2df303f4e1cb9088f1d00a4ed79124e871d3f427e832"
Sep 19 01:32:23.586794 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.586728    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4531bc1f125ee9506fed2df303f4e1cb9088f1d00a4ed79124e871d3f427e832"} err="failed to get container status \"4531bc1f125ee9506fed2df303f4e1cb9088f1d00a4ed79124e871d3f427e832\": rpc error: code = NotFound desc = could not find container \"4531bc1f125ee9506fed2df303f4e1cb9088f1d00a4ed79124e871d3f427e832\": container with ID starting with 4531bc1f125ee9506fed2df303f4e1cb9088f1d00a4ed79124e871d3f427e832 not found: ID does not exist"
Sep 19 01:32:23.586794 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.586752    2580 scope.go:117] "RemoveContainer" containerID="e6431e0ed2f9005cb4ce28537e2cc9babb6d62405550b37a223006f1d279d0dd"
Sep 19 01:32:23.586909 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.586885    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vtfs\" (UniqueName: \"kubernetes.io/projected/106fbc82-54ec-490b-9bf2-2613ca4e27ef-kube-api-access-2vtfs\") pod \"test-curl-7503-da9634\" (UID: \"106fbc82-54ec-490b-9bf2-2613ca4e27ef\") " pod="opendatahub/test-curl-7503-da9634"
Sep 19 01:32:23.594990 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.594974    2580 scope.go:117] "RemoveContainer" containerID="e6431e0ed2f9005cb4ce28537e2cc9babb6d62405550b37a223006f1d279d0dd"
Sep 19 01:32:23.595257 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:23.595237    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6431e0ed2f9005cb4ce28537e2cc9babb6d62405550b37a223006f1d279d0dd\": container with ID starting with e6431e0ed2f9005cb4ce28537e2cc9babb6d62405550b37a223006f1d279d0dd not found: ID does not exist" containerID="e6431e0ed2f9005cb4ce28537e2cc9babb6d62405550b37a223006f1d279d0dd"
Sep 19 01:32:23.595311 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.595264    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6431e0ed2f9005cb4ce28537e2cc9babb6d62405550b37a223006f1d279d0dd"} err="failed to get container status \"e6431e0ed2f9005cb4ce28537e2cc9babb6d62405550b37a223006f1d279d0dd\": rpc error: code = NotFound desc = could not find container \"e6431e0ed2f9005cb4ce28537e2cc9babb6d62405550b37a223006f1d279d0dd\": container with ID starting with e6431e0ed2f9005cb4ce28537e2cc9babb6d62405550b37a223006f1d279d0dd not found: ID does not exist"
Sep 19 01:32:23.679843 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.679815    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfslp\" (UniqueName: \"kubernetes.io/projected/5e0b0a82-a21a-4e31-8409-09c208370b98-kube-api-access-gfslp\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:32:23.679843 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.679841    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29965315-e87d-4bdb-aa4b-e8bc7ed5d275-plugins-config-volume\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:32:23.680025 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.679851    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e0b0a82-a21a-4e31-8409-09c208370b98-plugins-config-volume\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:32:23.680025 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.679861    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-276lz\" (UniqueName: \"kubernetes.io/projected/29965315-e87d-4bdb-aa4b-e8bc7ed5d275-kube-api-access-276lz\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:32:23.715148 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.715114    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-da9634"
Sep 19 01:32:23.887932 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.887910    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft"]
Sep 19 01:32:23.893653 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.893630    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c4ce82-6c94fb46c8-6pgft"]
Sep 19 01:32:23.909025 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.908996    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn"]
Sep 19 01:32:23.926114 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.926088    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c4ce82-5b766fc955-zmkzn"]
Sep 19 01:32:23.999943 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:23.999922    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7503-da9634"]
Sep 19 01:32:24.001960 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:32:24.001938    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod106fbc82_54ec_490b_9bf2_2613ca4e27ef.slice/crio-02431e225ae896912fceb9a9c2ffac1c4715376334fc871a8940f04262ba4e78 WatchSource:0}: Error finding container 02431e225ae896912fceb9a9c2ffac1c4715376334fc871a8940f04262ba4e78: Status 404 returned error can't find the container with id 02431e225ae896912fceb9a9c2ffac1c4715376334fc871a8940f04262ba4e78
Sep 19 01:32:24.574277 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:24.574251    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7503-da9634" event={"ID":"106fbc82-54ec-490b-9bf2-2613ca4e27ef","Type":"ContainerStarted","Data":"e9cdef2a5ad42601ebf4e21f5ee14a2dff0cd55572d426d9e08ebbdcb1dac7f8"}
Sep 19 01:32:24.574612 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:24.574292    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7503-da9634" event={"ID":"106fbc82-54ec-490b-9bf2-2613ca4e27ef","Type":"ContainerStarted","Data":"02431e225ae896912fceb9a9c2ffac1c4715376334fc871a8940f04262ba4e78"}
Sep 19 01:32:24.575430 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:24.575403    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7503-ab7bf8" podUID="14464d8a-681d-4b8e-8206-a4ed647e8aef" containerName="test-curl-7503-ab7bf8" containerID="cri-o://53bf62dbd5270ebae51f00219eb7c9ffa9bc9357fc1b049169bd3692b4867a49" gracePeriod=2
Sep 19 01:32:24.598904 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:24.598854    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7503-da9634" podStartSLOduration=1.598835233 podStartE2EDuration="1.598835233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:32:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:32:24.598363492 +0000 UTC m=+1199.505225541" watchObservedRunningTime="2026-09-19 01:32:24.598835233 +0000 UTC m=+1199.505697283"
Sep 19 01:32:24.600912 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:24.600876    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7503-ab7bf8\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="14464d8a-681d-4b8e-8206-a4ed647e8aef" pod="opendatahub/test-curl-7503-ab7bf8"
Sep 19 01:32:24.743924 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:24.743901    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-d8d841"
Sep 19 01:32:24.745992 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:24.745965    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7503-ab7bf8\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="14464d8a-681d-4b8e-8206-a4ed647e8aef" pod="opendatahub/test-curl-7503-ab7bf8"
Sep 19 01:32:24.792204 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:24.792183    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hgbh\" (UniqueName: \"kubernetes.io/projected/89e23b58-91b7-4481-9b0c-47c949439f3d-kube-api-access-6hgbh\") pod \"89e23b58-91b7-4481-9b0c-47c949439f3d\" (UID: \"89e23b58-91b7-4481-9b0c-47c949439f3d\") "
Sep 19 01:32:24.794341 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:24.794318    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89e23b58-91b7-4481-9b0c-47c949439f3d-kube-api-access-6hgbh" (OuterVolumeSpecName: "kube-api-access-6hgbh") pod "89e23b58-91b7-4481-9b0c-47c949439f3d" (UID: "89e23b58-91b7-4481-9b0c-47c949439f3d"). InnerVolumeSpecName "kube-api-access-6hgbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:32:24.893579 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:24.893509    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hgbh\" (UniqueName: \"kubernetes.io/projected/89e23b58-91b7-4481-9b0c-47c949439f3d-kube-api-access-6hgbh\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:32:25.064028 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:25.063999    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29965315-e87d-4bdb-aa4b-e8bc7ed5d275" path="/var/lib/kubelet/pods/29965315-e87d-4bdb-aa4b-e8bc7ed5d275/volumes"
Sep 19 01:32:25.064417 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:25.064403    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e0b0a82-a21a-4e31-8409-09c208370b98" path="/var/lib/kubelet/pods/5e0b0a82-a21a-4e31-8409-09c208370b98/volumes"
Sep 19 01:32:25.064774 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:25.064758    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89e23b58-91b7-4481-9b0c-47c949439f3d" path="/var/lib/kubelet/pods/89e23b58-91b7-4481-9b0c-47c949439f3d/volumes"
Sep 19 01:32:25.258781 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:25.258748    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7503-da9634"]
Sep 19 01:32:25.264273 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:25.264243    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7503-da9634"]
Sep 19 01:32:25.586621 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:25.586536    2580 generic.go:356] "Generic (PLEG): container finished" podID="89e23b58-91b7-4481-9b0c-47c949439f3d" containerID="f5a763f78932f0a9dbf278c13fb9594ec965026d1703b139d5dc0bfc49184a38" exitCode=137
Sep 19 01:32:25.586621 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:25.586590    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-d8d841"
Sep 19 01:32:25.587070 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:25.586661    2580 scope.go:117] "RemoveContainer" containerID="f5a763f78932f0a9dbf278c13fb9594ec965026d1703b139d5dc0bfc49184a38"
Sep 19 01:32:25.587070 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:25.586883    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7503-da9634" container="test-curl-7503-da9634"
Sep 19 01:32:25.596286 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:25.596263    2580 scope.go:117] "RemoveContainer" containerID="f5a763f78932f0a9dbf278c13fb9594ec965026d1703b139d5dc0bfc49184a38"
Sep 19 01:32:25.596568 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:25.596550    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5a763f78932f0a9dbf278c13fb9594ec965026d1703b139d5dc0bfc49184a38\": container with ID starting with f5a763f78932f0a9dbf278c13fb9594ec965026d1703b139d5dc0bfc49184a38 not found: ID does not exist" containerID="f5a763f78932f0a9dbf278c13fb9594ec965026d1703b139d5dc0bfc49184a38"
Sep 19 01:32:25.596631 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:25.596576    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5a763f78932f0a9dbf278c13fb9594ec965026d1703b139d5dc0bfc49184a38"} err="failed to get container status \"f5a763f78932f0a9dbf278c13fb9594ec965026d1703b139d5dc0bfc49184a38\": rpc error: code = NotFound desc = could not find container \"f5a763f78932f0a9dbf278c13fb9594ec965026d1703b139d5dc0bfc49184a38\": container with ID starting with f5a763f78932f0a9dbf278c13fb9594ec965026d1703b139d5dc0bfc49184a38 not found: ID does not exist"
Sep 19 01:32:26.594059 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:26.594029    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7503-da9634" podUID="106fbc82-54ec-490b-9bf2-2613ca4e27ef" containerName="test-curl-7503-da9634" containerID="cri-o://e9cdef2a5ad42601ebf4e21f5ee14a2dff0cd55572d426d9e08ebbdcb1dac7f8" gracePeriod=2
Sep 19 01:32:26.598169 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:26.598143    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7503-da9634\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="106fbc82-54ec-490b-9bf2-2613ca4e27ef" pod="opendatahub/test-curl-7503-da9634"
Sep 19 01:32:26.765522 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:26.765485    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-ab7bf8"
Sep 19 01:32:26.771109 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:26.771079    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7503-da9634\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="106fbc82-54ec-490b-9bf2-2613ca4e27ef" pod="opendatahub/test-curl-7503-da9634"
Sep 19 01:32:26.810810 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:26.810788    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t649t\" (UniqueName: \"kubernetes.io/projected/14464d8a-681d-4b8e-8206-a4ed647e8aef-kube-api-access-t649t\") pod \"14464d8a-681d-4b8e-8206-a4ed647e8aef\" (UID: \"14464d8a-681d-4b8e-8206-a4ed647e8aef\") "
Sep 19 01:32:26.812842 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:26.812818    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14464d8a-681d-4b8e-8206-a4ed647e8aef-kube-api-access-t649t" (OuterVolumeSpecName: "kube-api-access-t649t") pod "14464d8a-681d-4b8e-8206-a4ed647e8aef" (UID: "14464d8a-681d-4b8e-8206-a4ed647e8aef"). InnerVolumeSpecName "kube-api-access-t649t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:32:26.911869 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:26.911799    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t649t\" (UniqueName: \"kubernetes.io/projected/14464d8a-681d-4b8e-8206-a4ed647e8aef-kube-api-access-t649t\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:32:27.062351 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:27.062319    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7503-da9634\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="106fbc82-54ec-490b-9bf2-2613ca4e27ef" pod="opendatahub/test-curl-7503-da9634"
Sep 19 01:32:27.064616 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:27.064596    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14464d8a-681d-4b8e-8206-a4ed647e8aef" path="/var/lib/kubelet/pods/14464d8a-681d-4b8e-8206-a4ed647e8aef/volumes"
Sep 19 01:32:27.161280 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:27.161251    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f6bdh_74c8e2f6-1b01-4bd1-b491-ad407a15a109/console-operator/1.log"
Sep 19 01:32:27.164658 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:27.164587    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f6bdh_74c8e2f6-1b01-4bd1-b491-ad407a15a109/console-operator/1.log"
Sep 19 01:32:27.601898 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:27.601866    2580 generic.go:356] "Generic (PLEG): container finished" podID="14464d8a-681d-4b8e-8206-a4ed647e8aef" containerID="53bf62dbd5270ebae51f00219eb7c9ffa9bc9357fc1b049169bd3692b4867a49" exitCode=137
Sep 19 01:32:27.602344 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:27.601911    2580 scope.go:117] "RemoveContainer" containerID="53bf62dbd5270ebae51f00219eb7c9ffa9bc9357fc1b049169bd3692b4867a49"
Sep 19 01:32:27.602344 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:27.601910    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-ab7bf8"
Sep 19 01:32:27.611207 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:27.611187    2580 scope.go:117] "RemoveContainer" containerID="53bf62dbd5270ebae51f00219eb7c9ffa9bc9357fc1b049169bd3692b4867a49"
Sep 19 01:32:27.611512 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:27.611478    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53bf62dbd5270ebae51f00219eb7c9ffa9bc9357fc1b049169bd3692b4867a49\": container with ID starting with 53bf62dbd5270ebae51f00219eb7c9ffa9bc9357fc1b049169bd3692b4867a49 not found: ID does not exist" containerID="53bf62dbd5270ebae51f00219eb7c9ffa9bc9357fc1b049169bd3692b4867a49"
Sep 19 01:32:27.611614 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:27.611528    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53bf62dbd5270ebae51f00219eb7c9ffa9bc9357fc1b049169bd3692b4867a49"} err="failed to get container status \"53bf62dbd5270ebae51f00219eb7c9ffa9bc9357fc1b049169bd3692b4867a49\": rpc error: code = NotFound desc = could not find container \"53bf62dbd5270ebae51f00219eb7c9ffa9bc9357fc1b049169bd3692b4867a49\": container with ID starting with 53bf62dbd5270ebae51f00219eb7c9ffa9bc9357fc1b049169bd3692b4867a49 not found: ID does not exist"
Sep 19 01:32:28.787264 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:28.787242    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-da9634"
Sep 19 01:32:28.830438 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:28.830385    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vtfs\" (UniqueName: \"kubernetes.io/projected/106fbc82-54ec-490b-9bf2-2613ca4e27ef-kube-api-access-2vtfs\") pod \"106fbc82-54ec-490b-9bf2-2613ca4e27ef\" (UID: \"106fbc82-54ec-490b-9bf2-2613ca4e27ef\") "
Sep 19 01:32:28.832440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:28.832412    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/106fbc82-54ec-490b-9bf2-2613ca4e27ef-kube-api-access-2vtfs" (OuterVolumeSpecName: "kube-api-access-2vtfs") pod "106fbc82-54ec-490b-9bf2-2613ca4e27ef" (UID: "106fbc82-54ec-490b-9bf2-2613ca4e27ef"). InnerVolumeSpecName "kube-api-access-2vtfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:32:28.931740 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:28.931667    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vtfs\" (UniqueName: \"kubernetes.io/projected/106fbc82-54ec-490b-9bf2-2613ca4e27ef-kube-api-access-2vtfs\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:32:29.070860 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:29.070822    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="106fbc82-54ec-490b-9bf2-2613ca4e27ef" path="/var/lib/kubelet/pods/106fbc82-54ec-490b-9bf2-2613ca4e27ef/volumes"
Sep 19 01:32:29.617376 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:29.617344    2580 generic.go:356] "Generic (PLEG): container finished" podID="106fbc82-54ec-490b-9bf2-2613ca4e27ef" containerID="e9cdef2a5ad42601ebf4e21f5ee14a2dff0cd55572d426d9e08ebbdcb1dac7f8" exitCode=137
Sep 19 01:32:29.617573 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:29.617400    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-da9634"
Sep 19 01:32:29.617573 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:29.617453    2580 scope.go:117] "RemoveContainer" containerID="e9cdef2a5ad42601ebf4e21f5ee14a2dff0cd55572d426d9e08ebbdcb1dac7f8"
Sep 19 01:32:29.627084 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:29.627062    2580 scope.go:117] "RemoveContainer" containerID="e9cdef2a5ad42601ebf4e21f5ee14a2dff0cd55572d426d9e08ebbdcb1dac7f8"
Sep 19 01:32:29.627398 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:29.627376    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9cdef2a5ad42601ebf4e21f5ee14a2dff0cd55572d426d9e08ebbdcb1dac7f8\": container with ID starting with e9cdef2a5ad42601ebf4e21f5ee14a2dff0cd55572d426d9e08ebbdcb1dac7f8 not found: ID does not exist" containerID="e9cdef2a5ad42601ebf4e21f5ee14a2dff0cd55572d426d9e08ebbdcb1dac7f8"
Sep 19 01:32:29.627518 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:29.627404    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9cdef2a5ad42601ebf4e21f5ee14a2dff0cd55572d426d9e08ebbdcb1dac7f8"} err="failed to get container status \"e9cdef2a5ad42601ebf4e21f5ee14a2dff0cd55572d426d9e08ebbdcb1dac7f8\": rpc error: code = NotFound desc = could not find container \"e9cdef2a5ad42601ebf4e21f5ee14a2dff0cd55572d426d9e08ebbdcb1dac7f8\": container with ID starting with e9cdef2a5ad42601ebf4e21f5ee14a2dff0cd55572d426d9e08ebbdcb1dac7f8 not found: ID does not exist"
Sep 19 01:32:31.465285 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:31.465252    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14464d8a_681d_4b8e_8206_a4ed647e8aef.slice/crio-247d84df60f101eaacf0e83409670a3eebc543350966f4a06c0e20a54d1384e6\": RecentStats: unable to find data in memory cache]"
Sep 19 01:32:32.349600 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:32.349560    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"]
Sep 19 01:32:32.349780 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:32.349758    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46" podUID="3e8e948d-83e1-4202-880b-53611977d6cb" containerName="istio-proxy" containerID="cri-o://9729901154a2ae80afb4e903fa9c4ce8b87002bd37043784b6d771a9b118fed3" gracePeriod=30
Sep 19 01:32:33.998453 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:33.998423    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"]
Sep 19 01:32:33.998936 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:33.998921    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29965315-e87d-4bdb-aa4b-e8bc7ed5d275" containerName="payload-processing"
Sep 19 01:32:33.998936 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:33.998937    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="29965315-e87d-4bdb-aa4b-e8bc7ed5d275" containerName="payload-processing"
Sep 19 01:32:33.999050 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:33.998949    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="106fbc82-54ec-490b-9bf2-2613ca4e27ef" containerName="test-curl-7503-da9634"
Sep 19 01:32:33.999050 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:33.998955    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="106fbc82-54ec-490b-9bf2-2613ca4e27ef" containerName="test-curl-7503-da9634"
Sep 19 01:32:33.999050 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:33.998971    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e0b0a82-a21a-4e31-8409-09c208370b98" containerName="payload-pre-processing"
Sep 19 01:32:33.999050 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:33.998977    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e0b0a82-a21a-4e31-8409-09c208370b98" containerName="payload-pre-processing"
Sep 19 01:32:33.999182 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:33.999059    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="106fbc82-54ec-490b-9bf2-2613ca4e27ef" containerName="test-curl-7503-da9634"
Sep 19 01:32:33.999182 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:33.999069    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e0b0a82-a21a-4e31-8409-09c208370b98" containerName="payload-pre-processing"
Sep 19 01:32:33.999182 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:33.999077    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="29965315-e87d-4bdb-aa4b-e8bc7ed5d275" containerName="payload-processing"
Sep 19 01:32:34.003616 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:34.003594    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"
Sep 19 01:32:34.006256 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:34.006231    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-b5fe4519-serving-cert\""
Sep 19 01:32:34.014700 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:34.014676    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"]
Sep 19 01:32:34.087237 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:34.087207    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70fbeffe-5bd4-4821-a7c6-e596118f862e-openshift-service-ca\") pod \"maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7\" (UID: \"70fbeffe-5bd4-4821-a7c6-e596118f862e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"
Sep 19 01:32:34.087395 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:34.087254    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsz4z\" (UniqueName: \"kubernetes.io/projected/70fbeffe-5bd4-4821-a7c6-e596118f862e-kube-api-access-lsz4z\") pod \"maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7\" (UID: \"70fbeffe-5bd4-4821-a7c6-e596118f862e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"
Sep 19 01:32:34.087395 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:34.087355    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70fbeffe-5bd4-4821-a7c6-e596118f862e-metrics-certs\") pod \"maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7\" (UID: \"70fbeffe-5bd4-4821-a7c6-e596118f862e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"
Sep 19 01:32:34.087395 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:34.087386    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70fbeffe-5bd4-4821-a7c6-e596118f862e-maas-api-tls\") pod \"maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7\" (UID: \"70fbeffe-5bd4-4821-a7c6-e596118f862e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"
Sep 19 01:32:34.188160 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:34.188124    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70fbeffe-5bd4-4821-a7c6-e596118f862e-openshift-service-ca\") pod \"maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7\" (UID: \"70fbeffe-5bd4-4821-a7c6-e596118f862e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"
Sep 19 01:32:34.188319 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:34.188171    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsz4z\" (UniqueName: \"kubernetes.io/projected/70fbeffe-5bd4-4821-a7c6-e596118f862e-kube-api-access-lsz4z\") pod \"maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7\" (UID: \"70fbeffe-5bd4-4821-a7c6-e596118f862e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"
Sep 19 01:32:34.188319 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:34.188231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70fbeffe-5bd4-4821-a7c6-e596118f862e-metrics-certs\") pod \"maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7\" (UID: \"70fbeffe-5bd4-4821-a7c6-e596118f862e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"
Sep 19 01:32:34.188319 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:34.188251    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70fbeffe-5bd4-4821-a7c6-e596118f862e-maas-api-tls\") pod \"maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7\" (UID: \"70fbeffe-5bd4-4821-a7c6-e596118f862e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"
Sep 19 01:32:34.188424 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:34.188364    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-serving-cert: secret "maas-api-e2e-mt-b5fe4519-serving-cert" not found
Sep 19 01:32:34.188455 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:34.188431    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70fbeffe-5bd4-4821-a7c6-e596118f862e-maas-api-tls podName:70fbeffe-5bd4-4821-a7c6-e596118f862e nodeName:}" failed. No retries permitted until 2026-09-19 01:32:34.68841171 +0000 UTC m=+1209.595273754 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/70fbeffe-5bd4-4821-a7c6-e596118f862e-maas-api-tls") pod "maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7" (UID: "70fbeffe-5bd4-4821-a7c6-e596118f862e") : secret "maas-api-e2e-mt-b5fe4519-serving-cert" not found
Sep 19 01:32:34.188825 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:34.188797    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70fbeffe-5bd4-4821-a7c6-e596118f862e-openshift-service-ca\") pod \"maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7\" (UID: \"70fbeffe-5bd4-4821-a7c6-e596118f862e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"
Sep 19 01:32:34.190626 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:34.190608    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70fbeffe-5bd4-4821-a7c6-e596118f862e-metrics-certs\") pod \"maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7\" (UID: \"70fbeffe-5bd4-4821-a7c6-e596118f862e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"
Sep 19 01:32:34.200762 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:34.200739    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsz4z\" (UniqueName: \"kubernetes.io/projected/70fbeffe-5bd4-4821-a7c6-e596118f862e-kube-api-access-lsz4z\") pod \"maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7\" (UID: \"70fbeffe-5bd4-4821-a7c6-e596118f862e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"
Sep 19 01:32:34.287686 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:34.287618    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14464d8a_681d_4b8e_8206_a4ed647e8aef.slice/crio-247d84df60f101eaacf0e83409670a3eebc543350966f4a06c0e20a54d1384e6\": RecentStats: unable to find data in memory cache]"
Sep 19 01:32:34.693168 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:34.693074    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70fbeffe-5bd4-4821-a7c6-e596118f862e-maas-api-tls\") pod \"maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7\" (UID: \"70fbeffe-5bd4-4821-a7c6-e596118f862e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"
Sep 19 01:32:34.695360 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:34.695340    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70fbeffe-5bd4-4821-a7c6-e596118f862e-maas-api-tls\") pod \"maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7\" (UID: \"70fbeffe-5bd4-4821-a7c6-e596118f862e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"
Sep 19 01:32:34.916261 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:34.916229    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"
Sep 19 01:32:35.079050 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:35.079013    2580 patch_prober.go:28] interesting pod/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 19 01:32:35.079390 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:35.079067    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46" podUID="3e8e948d-83e1-4202-880b-53611977d6cb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 01:32:35.203085 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:35.203056    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"]
Sep 19 01:32:35.204487 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:32:35.204431    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70fbeffe_5bd4_4821_a7c6_e596118f862e.slice/crio-abb8fde56a1942859e9e761398ea21719c9a2854ebad6a9cce54d61e6a0fc097 WatchSource:0}: Error finding container abb8fde56a1942859e9e761398ea21719c9a2854ebad6a9cce54d61e6a0fc097: Status 404 returned error can't find the container with id abb8fde56a1942859e9e761398ea21719c9a2854ebad6a9cce54d61e6a0fc097
Sep 19 01:32:35.665436 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:35.665407    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7" event={"ID":"70fbeffe-5bd4-4821-a7c6-e596118f862e","Type":"ContainerStarted","Data":"abb8fde56a1942859e9e761398ea21719c9a2854ebad6a9cce54d61e6a0fc097"}
Sep 19 01:32:36.673586 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:36.673546    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7" event={"ID":"70fbeffe-5bd4-4821-a7c6-e596118f862e","Type":"ContainerStarted","Data":"c2aab50b90016652dd6cbafcd7a707651bb81363437ca3c3fe0f485c9636fbc5"}
Sep 19 01:32:36.674070 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:36.673667    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"
Sep 19 01:32:36.692640 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:36.692583    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7" podStartSLOduration=3.291516111 podStartE2EDuration="3.692563621s" totalImagesPullingTime="401.04751ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:32:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:32:36.689995325 +0000 UTC m=+1211.596857374" watchObservedRunningTime="2026-09-19 01:32:36.692563621 +0000 UTC m=+1211.599425671"
Sep 19 01:32:37.637369 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.637346    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:32:37.683956 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.683919    2580 generic.go:356] "Generic (PLEG): container finished" podID="3e8e948d-83e1-4202-880b-53611977d6cb" containerID="9729901154a2ae80afb4e903fa9c4ce8b87002bd37043784b6d771a9b118fed3" exitCode=0
Sep 19 01:32:37.684409 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.683961    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"
Sep 19 01:32:37.684409 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.683998    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46" event={"ID":"3e8e948d-83e1-4202-880b-53611977d6cb","Type":"ContainerDied","Data":"9729901154a2ae80afb4e903fa9c4ce8b87002bd37043784b6d771a9b118fed3"}
Sep 19 01:32:37.684409 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.684061    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46" event={"ID":"3e8e948d-83e1-4202-880b-53611977d6cb","Type":"ContainerDied","Data":"54dccf270e649b3fd6b85329acdd03f77087dc9910c99804f9cf193f2538778c"}
Sep 19 01:32:37.684409 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.684099    2580 scope.go:117] "RemoveContainer" containerID="9729901154a2ae80afb4e903fa9c4ce8b87002bd37043784b6d771a9b118fed3"
Sep 19 01:32:37.694199 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.694181    2580 scope.go:117] "RemoveContainer" containerID="daafa008927a902ceaa7611fd129356bc7da5ef95554b873628b20238674758f"
Sep 19 01:32:37.704407 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.704386    2580 scope.go:117] "RemoveContainer" containerID="9729901154a2ae80afb4e903fa9c4ce8b87002bd37043784b6d771a9b118fed3"
Sep 19 01:32:37.704746 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:37.704722    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9729901154a2ae80afb4e903fa9c4ce8b87002bd37043784b6d771a9b118fed3\": container with ID starting with 9729901154a2ae80afb4e903fa9c4ce8b87002bd37043784b6d771a9b118fed3 not found: ID does not exist" containerID="9729901154a2ae80afb4e903fa9c4ce8b87002bd37043784b6d771a9b118fed3"
Sep 19 01:32:37.704831 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.704759    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9729901154a2ae80afb4e903fa9c4ce8b87002bd37043784b6d771a9b118fed3"} err="failed to get container status \"9729901154a2ae80afb4e903fa9c4ce8b87002bd37043784b6d771a9b118fed3\": rpc error: code = NotFound desc = could not find container \"9729901154a2ae80afb4e903fa9c4ce8b87002bd37043784b6d771a9b118fed3\": container with ID starting with 9729901154a2ae80afb4e903fa9c4ce8b87002bd37043784b6d771a9b118fed3 not found: ID does not exist"
Sep 19 01:32:37.704831 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.704783    2580 scope.go:117] "RemoveContainer" containerID="daafa008927a902ceaa7611fd129356bc7da5ef95554b873628b20238674758f"
Sep 19 01:32:37.705082 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:37.705062    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daafa008927a902ceaa7611fd129356bc7da5ef95554b873628b20238674758f\": container with ID starting with daafa008927a902ceaa7611fd129356bc7da5ef95554b873628b20238674758f not found: ID does not exist" containerID="daafa008927a902ceaa7611fd129356bc7da5ef95554b873628b20238674758f"
Sep 19 01:32:37.705138 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.705089    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daafa008927a902ceaa7611fd129356bc7da5ef95554b873628b20238674758f"} err="failed to get container status \"daafa008927a902ceaa7611fd129356bc7da5ef95554b873628b20238674758f\": rpc error: code = NotFound desc = could not find container \"daafa008927a902ceaa7611fd129356bc7da5ef95554b873628b20238674758f\": container with ID starting with daafa008927a902ceaa7611fd129356bc7da5ef95554b873628b20238674758f not found: ID does not exist"
Sep 19 01:32:37.721135 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.721105    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e8e948d-83e1-4202-880b-53611977d6cb-istiod-ca-cert\") pod \"3e8e948d-83e1-4202-880b-53611977d6cb\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") "
Sep 19 01:32:37.721224 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.721167    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e8e948d-83e1-4202-880b-53611977d6cb-istio-podinfo\") pod \"3e8e948d-83e1-4202-880b-53611977d6cb\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") "
Sep 19 01:32:37.721224 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.721206    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-istio-data\") pod \"3e8e948d-83e1-4202-880b-53611977d6cb\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") "
Sep 19 01:32:37.721304 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.721251    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-credential-socket\") pod \"3e8e948d-83e1-4202-880b-53611977d6cb\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") "
Sep 19 01:32:37.721304 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.721287    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-istio-envoy\") pod \"3e8e948d-83e1-4202-880b-53611977d6cb\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") "
Sep 19 01:32:37.721398 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.721317    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d956h\" (UniqueName: \"kubernetes.io/projected/3e8e948d-83e1-4202-880b-53611977d6cb-kube-api-access-d956h\") pod \"3e8e948d-83e1-4202-880b-53611977d6cb\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") "
Sep 19 01:32:37.721398 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.721357    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e8e948d-83e1-4202-880b-53611977d6cb-istio-token\") pod \"3e8e948d-83e1-4202-880b-53611977d6cb\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") "
Sep 19 01:32:37.721516 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.721487    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e8e948d-83e1-4202-880b-53611977d6cb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3e8e948d-83e1-4202-880b-53611977d6cb" (UID: "3e8e948d-83e1-4202-880b-53611977d6cb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:32:37.721585 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.721518    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-workload-socket\") pod \"3e8e948d-83e1-4202-880b-53611977d6cb\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") "
Sep 19 01:32:37.721585 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.721575    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-workload-certs\") pod \"3e8e948d-83e1-4202-880b-53611977d6cb\" (UID: \"3e8e948d-83e1-4202-880b-53611977d6cb\") "
Sep 19 01:32:37.721688 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.721639    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3e8e948d-83e1-4202-880b-53611977d6cb" (UID: "3e8e948d-83e1-4202-880b-53611977d6cb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:32:37.721794 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.721770    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3e8e948d-83e1-4202-880b-53611977d6cb" (UID: "3e8e948d-83e1-4202-880b-53611977d6cb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:32:37.721900 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.721855    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-istio-data" (OuterVolumeSpecName: "istio-data") pod "3e8e948d-83e1-4202-880b-53611977d6cb" (UID: "3e8e948d-83e1-4202-880b-53611977d6cb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:32:37.721966 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.721940    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3e8e948d-83e1-4202-880b-53611977d6cb" (UID: "3e8e948d-83e1-4202-880b-53611977d6cb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:32:37.722058 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.722042    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-istio-data\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:32:37.722115 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.722063    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-credential-socket\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:32:37.722115 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.722082    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-workload-socket\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:32:37.722115 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.722096    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-workload-certs\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:32:37.722115 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.722108    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e8e948d-83e1-4202-880b-53611977d6cb-istiod-ca-cert\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:32:37.723492 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.723464    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3e8e948d-83e1-4202-880b-53611977d6cb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3e8e948d-83e1-4202-880b-53611977d6cb" (UID: "3e8e948d-83e1-4202-880b-53611977d6cb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 01:32:37.723649 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.723627    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e8e948d-83e1-4202-880b-53611977d6cb-istio-token" (OuterVolumeSpecName: "istio-token") pod "3e8e948d-83e1-4202-880b-53611977d6cb" (UID: "3e8e948d-83e1-4202-880b-53611977d6cb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:32:37.723883 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.723860    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3e8e948d-83e1-4202-880b-53611977d6cb" (UID: "3e8e948d-83e1-4202-880b-53611977d6cb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:32:37.723964 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.723906    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e8e948d-83e1-4202-880b-53611977d6cb-kube-api-access-d956h" (OuterVolumeSpecName: "kube-api-access-d956h") pod "3e8e948d-83e1-4202-880b-53611977d6cb" (UID: "3e8e948d-83e1-4202-880b-53611977d6cb"). InnerVolumeSpecName "kube-api-access-d956h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:32:37.823326 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.823294    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e8e948d-83e1-4202-880b-53611977d6cb-istio-envoy\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:32:37.823326 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.823325    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d956h\" (UniqueName: \"kubernetes.io/projected/3e8e948d-83e1-4202-880b-53611977d6cb-kube-api-access-d956h\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:32:37.823540 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.823406    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e8e948d-83e1-4202-880b-53611977d6cb-istio-token\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:32:37.823540 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:37.823421    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e8e948d-83e1-4202-880b-53611977d6cb-istio-podinfo\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:32:38.009639 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:38.009607    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"]
Sep 19 01:32:38.014026 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:38.014001    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-c4ce82-openshift-default-8555cd6c9b-bpw46"]
Sep 19 01:32:39.064515 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:39.064458    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e8e948d-83e1-4202-880b-53611977d6cb" path="/var/lib/kubelet/pods/3e8e948d-83e1-4202-880b-53611977d6cb/volumes"
Sep 19 01:32:40.508885 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:40.508849    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14464d8a_681d_4b8e_8206_a4ed647e8aef.slice/crio-247d84df60f101eaacf0e83409670a3eebc543350966f4a06c0e20a54d1384e6\": RecentStats: unable to find data in memory cache]"
Sep 19 01:32:40.509267 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:40.508848    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14464d8a_681d_4b8e_8206_a4ed647e8aef.slice/crio-247d84df60f101eaacf0e83409670a3eebc543350966f4a06c0e20a54d1384e6\": RecentStats: unable to find data in memory cache]"
Sep 19 01:32:41.476667 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:41.476629    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14464d8a_681d_4b8e_8206_a4ed647e8aef.slice/crio-247d84df60f101eaacf0e83409670a3eebc543350966f4a06c0e20a54d1384e6\": RecentStats: unable to find data in memory cache]"
Sep 19 01:32:42.689955 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:42.689928    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"
Sep 19 01:32:43.780030 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:43.779993    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"]
Sep 19 01:32:43.780557 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:43.780207    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p" podUID="918ea7e0-cbe8-4d4e-b1d9-9443339498bb" containerName="maas-api" containerID="cri-o://e3d91112809e9146f721b281b9c5d06199461544d867363f6e157aa39b5dc5bf" gracePeriod=30
Sep 19 01:32:43.830735 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:43.830706    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5a005d-29829690-t8bp4"]
Sep 19 01:32:43.838227 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:43.838199    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5a005d-29829690-t8bp4"]
Sep 19 01:32:43.906889 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:43.906852    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm"]
Sep 19 01:32:43.907132 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:43.907104    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm" podUID="6b4c2de6-e080-4411-a957-9d84d93cb263" containerName="payload-pre-processing" containerID="cri-o://4fdce4a462c2adab77e6fec94ea2e20d04fe188fb380173affc0f7d766fb4ce3" gracePeriod=30
Sep 19 01:32:44.075529 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.075491    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"
Sep 19 01:32:44.189357 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.189333    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-openshift-service-ca\") pod \"918ea7e0-cbe8-4d4e-b1d9-9443339498bb\" (UID: \"918ea7e0-cbe8-4d4e-b1d9-9443339498bb\") "
Sep 19 01:32:44.189486 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.189381    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-maas-api-tls\") pod \"918ea7e0-cbe8-4d4e-b1d9-9443339498bb\" (UID: \"918ea7e0-cbe8-4d4e-b1d9-9443339498bb\") "
Sep 19 01:32:44.189486 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.189476    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-metrics-certs\") pod \"918ea7e0-cbe8-4d4e-b1d9-9443339498bb\" (UID: \"918ea7e0-cbe8-4d4e-b1d9-9443339498bb\") "
Sep 19 01:32:44.189630 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.189603    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6ksn\" (UniqueName: \"kubernetes.io/projected/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-kube-api-access-f6ksn\") pod \"918ea7e0-cbe8-4d4e-b1d9-9443339498bb\" (UID: \"918ea7e0-cbe8-4d4e-b1d9-9443339498bb\") "
Sep 19 01:32:44.189767 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.189745    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "918ea7e0-cbe8-4d4e-b1d9-9443339498bb" (UID: "918ea7e0-cbe8-4d4e-b1d9-9443339498bb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:32:44.190050 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.190030    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-openshift-service-ca\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:32:44.190574 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.190548    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm"
Sep 19 01:32:44.191769 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.191742    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "918ea7e0-cbe8-4d4e-b1d9-9443339498bb" (UID: "918ea7e0-cbe8-4d4e-b1d9-9443339498bb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:32:44.191839 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.191766    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "918ea7e0-cbe8-4d4e-b1d9-9443339498bb" (UID: "918ea7e0-cbe8-4d4e-b1d9-9443339498bb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:32:44.191839 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.191795    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-kube-api-access-f6ksn" (OuterVolumeSpecName: "kube-api-access-f6ksn") pod "918ea7e0-cbe8-4d4e-b1d9-9443339498bb" (UID: "918ea7e0-cbe8-4d4e-b1d9-9443339498bb"). InnerVolumeSpecName "kube-api-access-f6ksn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:32:44.290832 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.290790    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b4c2de6-e080-4411-a957-9d84d93cb263-plugins-config-volume\") pod \"6b4c2de6-e080-4411-a957-9d84d93cb263\" (UID: \"6b4c2de6-e080-4411-a957-9d84d93cb263\") "
Sep 19 01:32:44.290986 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.290940    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mvqr\" (UniqueName: \"kubernetes.io/projected/6b4c2de6-e080-4411-a957-9d84d93cb263-kube-api-access-7mvqr\") pod \"6b4c2de6-e080-4411-a957-9d84d93cb263\" (UID: \"6b4c2de6-e080-4411-a957-9d84d93cb263\") "
Sep 19 01:32:44.291178 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.291153    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b4c2de6-e080-4411-a957-9d84d93cb263-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6b4c2de6-e080-4411-a957-9d84d93cb263" (UID: "6b4c2de6-e080-4411-a957-9d84d93cb263"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:32:44.291298 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.291283    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-maas-api-tls\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:32:44.291346 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.291300    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-metrics-certs\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:32:44.291346 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.291315    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b4c2de6-e080-4411-a957-9d84d93cb263-plugins-config-volume\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:32:44.291346 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.291335    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6ksn\" (UniqueName: \"kubernetes.io/projected/918ea7e0-cbe8-4d4e-b1d9-9443339498bb-kube-api-access-f6ksn\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:32:44.293034 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.293008    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b4c2de6-e080-4411-a957-9d84d93cb263-kube-api-access-7mvqr" (OuterVolumeSpecName: "kube-api-access-7mvqr") pod "6b4c2de6-e080-4411-a957-9d84d93cb263" (UID: "6b4c2de6-e080-4411-a957-9d84d93cb263"). InnerVolumeSpecName "kube-api-access-7mvqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:32:44.392349 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.392266    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mvqr\" (UniqueName: \"kubernetes.io/projected/6b4c2de6-e080-4411-a957-9d84d93cb263-kube-api-access-7mvqr\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:32:44.735986 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.735954    2580 generic.go:356] "Generic (PLEG): container finished" podID="6b4c2de6-e080-4411-a957-9d84d93cb263" containerID="4fdce4a462c2adab77e6fec94ea2e20d04fe188fb380173affc0f7d766fb4ce3" exitCode=0
Sep 19 01:32:44.736187 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.736017    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm"
Sep 19 01:32:44.736187 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.736048    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm" event={"ID":"6b4c2de6-e080-4411-a957-9d84d93cb263","Type":"ContainerDied","Data":"4fdce4a462c2adab77e6fec94ea2e20d04fe188fb380173affc0f7d766fb4ce3"}
Sep 19 01:32:44.736187 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.736152    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm" event={"ID":"6b4c2de6-e080-4411-a957-9d84d93cb263","Type":"ContainerDied","Data":"08e4f55a368202b1be6b80d06e885ba3b75a70d8831086c98aa21bd3ddbdee4e"}
Sep 19 01:32:44.736367 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.736188    2580 scope.go:117] "RemoveContainer" containerID="4fdce4a462c2adab77e6fec94ea2e20d04fe188fb380173affc0f7d766fb4ce3"
Sep 19 01:32:44.737899 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.737873    2580 generic.go:356] "Generic (PLEG): container finished" podID="918ea7e0-cbe8-4d4e-b1d9-9443339498bb" containerID="e3d91112809e9146f721b281b9c5d06199461544d867363f6e157aa39b5dc5bf" exitCode=0
Sep 19 01:32:44.738017 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.737926    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"
Sep 19 01:32:44.738017 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.737956    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p" event={"ID":"918ea7e0-cbe8-4d4e-b1d9-9443339498bb","Type":"ContainerDied","Data":"e3d91112809e9146f721b281b9c5d06199461544d867363f6e157aa39b5dc5bf"}
Sep 19 01:32:44.738144 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.738034    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p" event={"ID":"918ea7e0-cbe8-4d4e-b1d9-9443339498bb","Type":"ContainerDied","Data":"5ae29c498f98f3d5bfaba488525205c73173f13eda96d375fc903faff1e4b341"}
Sep 19 01:32:44.747978 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.747880    2580 scope.go:117] "RemoveContainer" containerID="4fdce4a462c2adab77e6fec94ea2e20d04fe188fb380173affc0f7d766fb4ce3"
Sep 19 01:32:44.748358 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:44.748330    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fdce4a462c2adab77e6fec94ea2e20d04fe188fb380173affc0f7d766fb4ce3\": container with ID starting with 4fdce4a462c2adab77e6fec94ea2e20d04fe188fb380173affc0f7d766fb4ce3 not found: ID does not exist" containerID="4fdce4a462c2adab77e6fec94ea2e20d04fe188fb380173affc0f7d766fb4ce3"
Sep 19 01:32:44.748438 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.748369    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fdce4a462c2adab77e6fec94ea2e20d04fe188fb380173affc0f7d766fb4ce3"} err="failed to get container status \"4fdce4a462c2adab77e6fec94ea2e20d04fe188fb380173affc0f7d766fb4ce3\": rpc error: code = NotFound desc = could not find container \"4fdce4a462c2adab77e6fec94ea2e20d04fe188fb380173affc0f7d766fb4ce3\": container with ID starting with 4fdce4a462c2adab77e6fec94ea2e20d04fe188fb380173affc0f7d766fb4ce3 not found: ID does not exist"
Sep 19 01:32:44.748438 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.748390    2580 scope.go:117] "RemoveContainer" containerID="e3d91112809e9146f721b281b9c5d06199461544d867363f6e157aa39b5dc5bf"
Sep 19 01:32:44.757902 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.757887    2580 scope.go:117] "RemoveContainer" containerID="e3d91112809e9146f721b281b9c5d06199461544d867363f6e157aa39b5dc5bf"
Sep 19 01:32:44.758135 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:44.758117    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3d91112809e9146f721b281b9c5d06199461544d867363f6e157aa39b5dc5bf\": container with ID starting with e3d91112809e9146f721b281b9c5d06199461544d867363f6e157aa39b5dc5bf not found: ID does not exist" containerID="e3d91112809e9146f721b281b9c5d06199461544d867363f6e157aa39b5dc5bf"
Sep 19 01:32:44.758181 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.758140    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3d91112809e9146f721b281b9c5d06199461544d867363f6e157aa39b5dc5bf"} err="failed to get container status \"e3d91112809e9146f721b281b9c5d06199461544d867363f6e157aa39b5dc5bf\": rpc error: code = NotFound desc = could not find container \"e3d91112809e9146f721b281b9c5d06199461544d867363f6e157aa39b5dc5bf\": container with ID starting with e3d91112809e9146f721b281b9c5d06199461544d867363f6e157aa39b5dc5bf not found: ID does not exist"
Sep 19 01:32:44.773819 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.773788    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm"]
Sep 19 01:32:44.782305 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.782286    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5a005d-6cfdc9db6-6nhxm"]
Sep 19 01:32:44.806155 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.806120    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"]
Sep 19 01:32:44.810401 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:44.810381    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5a005d-778f9dc94b-w8z8p"]
Sep 19 01:32:45.064716 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:45.064631    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b2c5a78-c92d-4a03-ac8a-594a80841ac1" path="/var/lib/kubelet/pods/3b2c5a78-c92d-4a03-ac8a-594a80841ac1/volumes"
Sep 19 01:32:45.064996 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:45.064982    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b4c2de6-e080-4411-a957-9d84d93cb263" path="/var/lib/kubelet/pods/6b4c2de6-e080-4411-a957-9d84d93cb263/volumes"
Sep 19 01:32:45.065336 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:32:45.065324    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="918ea7e0-cbe8-4d4e-b1d9-9443339498bb" path="/var/lib/kubelet/pods/918ea7e0-cbe8-4d4e-b1d9-9443339498bb/volumes"
Sep 19 01:32:49.318255 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:49.318224    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14464d8a_681d_4b8e_8206_a4ed647e8aef.slice/crio-247d84df60f101eaacf0e83409670a3eebc543350966f4a06c0e20a54d1384e6\": RecentStats: unable to find data in memory cache]"
Sep 19 01:32:51.488206 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:32:51.488172    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14464d8a_681d_4b8e_8206_a4ed647e8aef.slice/crio-247d84df60f101eaacf0e83409670a3eebc543350966f4a06c0e20a54d1384e6\": RecentStats: unable to find data in memory cache]"
Sep 19 01:33:00.234808 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.234762    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"]
Sep 19 01:33:00.235480 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.235462    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b4c2de6-e080-4411-a957-9d84d93cb263" containerName="payload-pre-processing"
Sep 19 01:33:00.235602 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.235482    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b4c2de6-e080-4411-a957-9d84d93cb263" containerName="payload-pre-processing"
Sep 19 01:33:00.235602 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.235525    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="918ea7e0-cbe8-4d4e-b1d9-9443339498bb" containerName="maas-api"
Sep 19 01:33:00.235602 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.235535    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="918ea7e0-cbe8-4d4e-b1d9-9443339498bb" containerName="maas-api"
Sep 19 01:33:00.235602 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.235554    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e8e948d-83e1-4202-880b-53611977d6cb" containerName="istio-proxy"
Sep 19 01:33:00.235602 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.235563    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e8e948d-83e1-4202-880b-53611977d6cb" containerName="istio-proxy"
Sep 19 01:33:00.235602 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.235581    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e8e948d-83e1-4202-880b-53611977d6cb" containerName="istio-proxy"
Sep 19 01:33:00.235602 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.235590    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e8e948d-83e1-4202-880b-53611977d6cb" containerName="istio-proxy"
Sep 19 01:33:00.235974 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.235709    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e8e948d-83e1-4202-880b-53611977d6cb" containerName="istio-proxy"
Sep 19 01:33:00.235974 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.235723    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e8e948d-83e1-4202-880b-53611977d6cb" containerName="istio-proxy"
Sep 19 01:33:00.235974 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.235735    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b4c2de6-e080-4411-a957-9d84d93cb263" containerName="payload-pre-processing"
Sep 19 01:33:00.235974 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.235746    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="918ea7e0-cbe8-4d4e-b1d9-9443339498bb" containerName="maas-api"
Sep 19 01:33:00.239666 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.239641    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"
Sep 19 01:33:00.241231 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.241200    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nscnr\" (UniqueName: \"kubernetes.io/projected/65331bea-956d-4ab0-aff8-ca56a9f6b517-kube-api-access-nscnr\") pod \"maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr\" (UID: \"65331bea-956d-4ab0-aff8-ca56a9f6b517\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"
Sep 19 01:33:00.241349 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.241260    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65331bea-956d-4ab0-aff8-ca56a9f6b517-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr\" (UID: \"65331bea-956d-4ab0-aff8-ca56a9f6b517\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"
Sep 19 01:33:00.241349 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.241328    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65331bea-956d-4ab0-aff8-ca56a9f6b517-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr\" (UID: \"65331bea-956d-4ab0-aff8-ca56a9f6b517\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"
Sep 19 01:33:00.241439 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.241382    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65331bea-956d-4ab0-aff8-ca56a9f6b517-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr\" (UID: \"65331bea-956d-4ab0-aff8-ca56a9f6b517\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"
Sep 19 01:33:00.245346 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.245323    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-751f6a-serving-cert\""
Sep 19 01:33:00.258440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.258416    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"]
Sep 19 01:33:00.342315 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.342285    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65331bea-956d-4ab0-aff8-ca56a9f6b517-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr\" (UID: \"65331bea-956d-4ab0-aff8-ca56a9f6b517\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"
Sep 19 01:33:00.342506 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.342348    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nscnr\" (UniqueName: \"kubernetes.io/projected/65331bea-956d-4ab0-aff8-ca56a9f6b517-kube-api-access-nscnr\") pod \"maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr\" (UID: \"65331bea-956d-4ab0-aff8-ca56a9f6b517\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"
Sep 19 01:33:00.342506 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.342378    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65331bea-956d-4ab0-aff8-ca56a9f6b517-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr\" (UID: \"65331bea-956d-4ab0-aff8-ca56a9f6b517\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"
Sep 19 01:33:00.342506 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.342403    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65331bea-956d-4ab0-aff8-ca56a9f6b517-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr\" (UID: \"65331bea-956d-4ab0-aff8-ca56a9f6b517\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"
Sep 19 01:33:00.342699 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:33:00.342550    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-serving-cert: secret "maas-api-e2e-shared-a-w3-751f6a-serving-cert" not found
Sep 19 01:33:00.342699 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:33:00.342621    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65331bea-956d-4ab0-aff8-ca56a9f6b517-maas-api-tls podName:65331bea-956d-4ab0-aff8-ca56a9f6b517 nodeName:}" failed. No retries permitted until 2026-09-19 01:33:00.842599063 +0000 UTC m=+1235.749461109 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/65331bea-956d-4ab0-aff8-ca56a9f6b517-maas-api-tls") pod "maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr" (UID: "65331bea-956d-4ab0-aff8-ca56a9f6b517") : secret "maas-api-e2e-shared-a-w3-751f6a-serving-cert" not found
Sep 19 01:33:00.342993 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.342972    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65331bea-956d-4ab0-aff8-ca56a9f6b517-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr\" (UID: \"65331bea-956d-4ab0-aff8-ca56a9f6b517\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"
Sep 19 01:33:00.344799 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.344782    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65331bea-956d-4ab0-aff8-ca56a9f6b517-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr\" (UID: \"65331bea-956d-4ab0-aff8-ca56a9f6b517\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"
Sep 19 01:33:00.358574 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.358547    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nscnr\" (UniqueName: \"kubernetes.io/projected/65331bea-956d-4ab0-aff8-ca56a9f6b517-kube-api-access-nscnr\") pod \"maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr\" (UID: \"65331bea-956d-4ab0-aff8-ca56a9f6b517\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"
Sep 19 01:33:00.846723 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.846690    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65331bea-956d-4ab0-aff8-ca56a9f6b517-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr\" (UID: \"65331bea-956d-4ab0-aff8-ca56a9f6b517\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"
Sep 19 01:33:00.849199 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.849178    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65331bea-956d-4ab0-aff8-ca56a9f6b517-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr\" (UID: \"65331bea-956d-4ab0-aff8-ca56a9f6b517\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"
Sep 19 01:33:00.851944 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:00.851930    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"
Sep 19 01:33:01.167178 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:01.167033    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"]
Sep 19 01:33:01.169728 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:33:01.169699    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65331bea_956d_4ab0_aff8_ca56a9f6b517.slice/crio-73b31d252aea9b99fa1cded62daf6f0118a520c1a45aca08517472cabb0468f2 WatchSource:0}: Error finding container 73b31d252aea9b99fa1cded62daf6f0118a520c1a45aca08517472cabb0468f2: Status 404 returned error can't find the container with id 73b31d252aea9b99fa1cded62daf6f0118a520c1a45aca08517472cabb0468f2
Sep 19 01:33:01.554963 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:33:01.554933    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14464d8a_681d_4b8e_8206_a4ed647e8aef.slice/crio-247d84df60f101eaacf0e83409670a3eebc543350966f4a06c0e20a54d1384e6\": RecentStats: unable to find data in memory cache]"
Sep 19 01:33:01.855243 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:01.855207    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr" event={"ID":"65331bea-956d-4ab0-aff8-ca56a9f6b517","Type":"ContainerStarted","Data":"449bc2d3267f87bb94a8c467c77abfdf4bb4283a572efac9cb18e6eb7069c836"}
Sep 19 01:33:01.855424 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:01.855266    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr" event={"ID":"65331bea-956d-4ab0-aff8-ca56a9f6b517","Type":"ContainerStarted","Data":"73b31d252aea9b99fa1cded62daf6f0118a520c1a45aca08517472cabb0468f2"}
Sep 19 01:33:01.855469 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:01.855419    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"
Sep 19 01:33:01.888966 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:01.888900    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr" podStartSLOduration=1.578586134 podStartE2EDuration="1.888878957s" totalImagesPullingTime="310.292823ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:33:01.88768322 +0000 UTC m=+1236.794545281" watchObservedRunningTime="2026-09-19 01:33:01.888878957 +0000 UTC m=+1236.795741004"
Sep 19 01:33:04.292516 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:33:04.292464    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14464d8a_681d_4b8e_8206_a4ed647e8aef.slice/crio-247d84df60f101eaacf0e83409670a3eebc543350966f4a06c0e20a54d1384e6\": RecentStats: unable to find data in memory cache]"
Sep 19 01:33:07.867685 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:07.867659    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"
Sep 19 01:33:11.612578 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:33:11.612509    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14464d8a_681d_4b8e_8206_a4ed647e8aef.slice/crio-247d84df60f101eaacf0e83409670a3eebc543350966f4a06c0e20a54d1384e6\": RecentStats: unable to find data in memory cache]"
Sep 19 01:33:19.319369 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:33:19.319337    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14464d8a_681d_4b8e_8206_a4ed647e8aef.slice/crio-247d84df60f101eaacf0e83409670a3eebc543350966f4a06c0e20a54d1384e6\": RecentStats: unable to find data in memory cache]"
Sep 19 01:33:21.625209 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:33:21.625155    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14464d8a_681d_4b8e_8206_a4ed647e8aef.slice/crio-247d84df60f101eaacf0e83409670a3eebc543350966f4a06c0e20a54d1384e6\": RecentStats: unable to find data in memory cache]"
Sep 19 01:33:32.517887 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:32.517828    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"]
Sep 19 01:33:32.523282 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:32.523260    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"
Sep 19 01:33:32.526452 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:32.526426    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-bebef4-serving-cert\""
Sep 19 01:33:32.550469 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:32.550437    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"]
Sep 19 01:33:32.643343 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:32.643310    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmcln\" (UniqueName: \"kubernetes.io/projected/f39221aa-43db-4aba-871e-0d285e983c73-kube-api-access-pmcln\") pod \"maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq\" (UID: \"f39221aa-43db-4aba-871e-0d285e983c73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"
Sep 19 01:33:32.643584 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:32.643371    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f39221aa-43db-4aba-871e-0d285e983c73-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq\" (UID: \"f39221aa-43db-4aba-871e-0d285e983c73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"
Sep 19 01:33:32.643584 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:32.643469    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f39221aa-43db-4aba-871e-0d285e983c73-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq\" (UID: \"f39221aa-43db-4aba-871e-0d285e983c73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"
Sep 19 01:33:32.643584 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:32.643539    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f39221aa-43db-4aba-871e-0d285e983c73-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq\" (UID: \"f39221aa-43db-4aba-871e-0d285e983c73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"
Sep 19 01:33:32.744448 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:32.744411    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmcln\" (UniqueName: \"kubernetes.io/projected/f39221aa-43db-4aba-871e-0d285e983c73-kube-api-access-pmcln\") pod \"maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq\" (UID: \"f39221aa-43db-4aba-871e-0d285e983c73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"
Sep 19 01:33:32.744633 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:32.744467    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f39221aa-43db-4aba-871e-0d285e983c73-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq\" (UID: \"f39221aa-43db-4aba-871e-0d285e983c73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"
Sep 19 01:33:32.744633 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:32.744521    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f39221aa-43db-4aba-871e-0d285e983c73-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq\" (UID: \"f39221aa-43db-4aba-871e-0d285e983c73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"
Sep 19 01:33:32.744633 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:32.744554    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f39221aa-43db-4aba-871e-0d285e983c73-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq\" (UID: \"f39221aa-43db-4aba-871e-0d285e983c73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"
Sep 19 01:33:32.744797 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:33:32.744651    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-serving-cert: secret "maas-api-e2e-shared-b-w3-bebef4-serving-cert" not found
Sep 19 01:33:32.744797 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:33:32.744720    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f39221aa-43db-4aba-871e-0d285e983c73-maas-api-tls podName:f39221aa-43db-4aba-871e-0d285e983c73 nodeName:}" failed. No retries permitted until 2026-09-19 01:33:33.24469709 +0000 UTC m=+1268.151559121 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f39221aa-43db-4aba-871e-0d285e983c73-maas-api-tls") pod "maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq" (UID: "f39221aa-43db-4aba-871e-0d285e983c73") : secret "maas-api-e2e-shared-b-w3-bebef4-serving-cert" not found
Sep 19 01:33:32.745320 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:32.745293    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f39221aa-43db-4aba-871e-0d285e983c73-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq\" (UID: \"f39221aa-43db-4aba-871e-0d285e983c73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"
Sep 19 01:33:32.747051 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:32.747032    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f39221aa-43db-4aba-871e-0d285e983c73-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq\" (UID: \"f39221aa-43db-4aba-871e-0d285e983c73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"
Sep 19 01:33:32.755260 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:32.755234    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmcln\" (UniqueName: \"kubernetes.io/projected/f39221aa-43db-4aba-871e-0d285e983c73-kube-api-access-pmcln\") pod \"maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq\" (UID: \"f39221aa-43db-4aba-871e-0d285e983c73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"
Sep 19 01:33:33.249635 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:33.249603    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f39221aa-43db-4aba-871e-0d285e983c73-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq\" (UID: \"f39221aa-43db-4aba-871e-0d285e983c73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"
Sep 19 01:33:33.252123 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:33.252091    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f39221aa-43db-4aba-871e-0d285e983c73-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq\" (UID: \"f39221aa-43db-4aba-871e-0d285e983c73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"
Sep 19 01:33:33.397605 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:33.397572    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh"]
Sep 19 01:33:33.401657 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:33.401641    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh"
Sep 19 01:33:33.405641 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:33.405618    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-bebef4\""
Sep 19 01:33:33.406438 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:33.406417    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-bebef4-dockercfg-s4ppb\""
Sep 19 01:33:33.417125 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:33.417104    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh"]
Sep 19 01:33:33.437249 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:33.437218    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"
Sep 19 01:33:33.553466 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:33.553378    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk58c\" (UniqueName: \"kubernetes.io/projected/a3adc67a-5992-497a-995b-7e855970367b-kube-api-access-zk58c\") pod \"payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh\" (UID: \"a3adc67a-5992-497a-995b-7e855970367b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh"
Sep 19 01:33:33.553828 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:33.553572    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3adc67a-5992-497a-995b-7e855970367b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh\" (UID: \"a3adc67a-5992-497a-995b-7e855970367b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh"
Sep 19 01:33:33.654223 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:33.654184    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3adc67a-5992-497a-995b-7e855970367b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh\" (UID: \"a3adc67a-5992-497a-995b-7e855970367b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh"
Sep 19 01:33:33.654415 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:33.654255    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk58c\" (UniqueName: \"kubernetes.io/projected/a3adc67a-5992-497a-995b-7e855970367b-kube-api-access-zk58c\") pod \"payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh\" (UID: \"a3adc67a-5992-497a-995b-7e855970367b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh"
Sep 19 01:33:33.655038 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:33.655001    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3adc67a-5992-497a-995b-7e855970367b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh\" (UID: \"a3adc67a-5992-497a-995b-7e855970367b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh"
Sep 19 01:33:33.663408 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:33.663388    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk58c\" (UniqueName: \"kubernetes.io/projected/a3adc67a-5992-497a-995b-7e855970367b-kube-api-access-zk58c\") pod \"payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh\" (UID: \"a3adc67a-5992-497a-995b-7e855970367b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh"
Sep 19 01:33:33.713449 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:33.713398    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh"
Sep 19 01:33:33.717194 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:33.717167    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"]
Sep 19 01:33:33.719013 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:33:33.718984    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf39221aa_43db_4aba_871e_0d285e983c73.slice/crio-29958c09a3dd20224c54e514a7ad6ac0236d74af7b48ba91c328445418f15ca0 WatchSource:0}: Error finding container 29958c09a3dd20224c54e514a7ad6ac0236d74af7b48ba91c328445418f15ca0: Status 404 returned error can't find the container with id 29958c09a3dd20224c54e514a7ad6ac0236d74af7b48ba91c328445418f15ca0
Sep 19 01:33:34.006410 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:34.006377    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh"]
Sep 19 01:33:34.007624 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:33:34.007584    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3adc67a_5992_497a_995b_7e855970367b.slice/crio-fa7ac07cec66793340ce97baa338da7129030b5223c0d72c5a83efb7e1e72f6a WatchSource:0}: Error finding container fa7ac07cec66793340ce97baa338da7129030b5223c0d72c5a83efb7e1e72f6a: Status 404 returned error can't find the container with id fa7ac07cec66793340ce97baa338da7129030b5223c0d72c5a83efb7e1e72f6a
Sep 19 01:33:34.071254 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:34.071227    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh" event={"ID":"a3adc67a-5992-497a-995b-7e855970367b","Type":"ContainerStarted","Data":"fa7ac07cec66793340ce97baa338da7129030b5223c0d72c5a83efb7e1e72f6a"}
Sep 19 01:33:34.072476 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:34.072452    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq" event={"ID":"f39221aa-43db-4aba-871e-0d285e983c73","Type":"ContainerStarted","Data":"29958c09a3dd20224c54e514a7ad6ac0236d74af7b48ba91c328445418f15ca0"}
Sep 19 01:33:35.080410 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:35.080370    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq" event={"ID":"f39221aa-43db-4aba-871e-0d285e983c73","Type":"ContainerStarted","Data":"717633fe1deb0c8b420c8298057f896aa6992b11edc9928817e091f574b447f1"}
Sep 19 01:33:35.080869 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:35.080439    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"
Sep 19 01:33:35.082097 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:35.082073    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh" event={"ID":"a3adc67a-5992-497a-995b-7e855970367b","Type":"ContainerStarted","Data":"ccc6c234855316c34b0e8cf01e10c551752379ac1d032f8ce6af6a190a1a82dd"}
Sep 19 01:33:35.082210 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:35.082135    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh"
Sep 19 01:33:35.102611 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:35.102565    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq" podStartSLOduration=2.788382029 podStartE2EDuration="3.102548922s" totalImagesPullingTime="314.166893ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:33:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:33:35.098812654 +0000 UTC m=+1270.005674701" watchObservedRunningTime="2026-09-19 01:33:35.102548922 +0000 UTC m=+1270.009410970"
Sep 19 01:33:35.122933 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:35.122885    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh" podStartSLOduration=2.122869983 podStartE2EDuration="2.122869983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:33:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:33:35.119127709 +0000 UTC m=+1270.025989757" watchObservedRunningTime="2026-09-19 01:33:35.122869983 +0000 UTC m=+1270.029732030"
Sep 19 01:33:41.094784 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:41.094756    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"
Sep 19 01:33:46.091548 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:33:46.091516    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh"
Sep 19 01:34:15.244709 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.244674    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"]
Sep 19 01:34:15.249768 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.249749    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:34:15.275519 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.275478    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-751f6a\"/\"kube-root-ca.crt\""
Sep 19 01:34:15.278316 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.278294    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-751f6a\"/\"auth-test-model-751f6a-kserve-self-signed-certs\""
Sep 19 01:34:15.279802 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.279781    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-751f6a\"/\"default-dockercfg-nxcm6\""
Sep 19 01:34:15.280216 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.280197    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-751f6a\"/\"openshift-service-ca.crt\""
Sep 19 01:34:15.281962 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.281940    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"]
Sep 19 01:34:15.372002 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.371967    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33f30bb4-e89e-4c99-9801-787321b0a13d-model-cache\") pod \"auth-test-model-751f6a-kserve-888dfd979-fb5jh\" (UID: \"33f30bb4-e89e-4c99-9801-787321b0a13d\") " pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:34:15.372198 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.372099    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33f30bb4-e89e-4c99-9801-787321b0a13d-dshm\") pod \"auth-test-model-751f6a-kserve-888dfd979-fb5jh\" (UID: \"33f30bb4-e89e-4c99-9801-787321b0a13d\") " pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:34:15.372198 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.372123    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33f30bb4-e89e-4c99-9801-787321b0a13d-tmp-dir\") pod \"auth-test-model-751f6a-kserve-888dfd979-fb5jh\" (UID: \"33f30bb4-e89e-4c99-9801-787321b0a13d\") " pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:34:15.372198 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.372147    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33f30bb4-e89e-4c99-9801-787321b0a13d-home\") pod \"auth-test-model-751f6a-kserve-888dfd979-fb5jh\" (UID: \"33f30bb4-e89e-4c99-9801-787321b0a13d\") " pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:34:15.372347 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.372208    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33f30bb4-e89e-4c99-9801-787321b0a13d-tls-certs\") pod \"auth-test-model-751f6a-kserve-888dfd979-fb5jh\" (UID: \"33f30bb4-e89e-4c99-9801-787321b0a13d\") " pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:34:15.372347 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.372302    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68szp\" (UniqueName: \"kubernetes.io/projected/33f30bb4-e89e-4c99-9801-787321b0a13d-kube-api-access-68szp\") pod \"auth-test-model-751f6a-kserve-888dfd979-fb5jh\" (UID: \"33f30bb4-e89e-4c99-9801-787321b0a13d\") " pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:34:15.473575 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.473541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33f30bb4-e89e-4c99-9801-787321b0a13d-model-cache\") pod \"auth-test-model-751f6a-kserve-888dfd979-fb5jh\" (UID: \"33f30bb4-e89e-4c99-9801-787321b0a13d\") " pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:34:15.473776 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.473632    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33f30bb4-e89e-4c99-9801-787321b0a13d-dshm\") pod \"auth-test-model-751f6a-kserve-888dfd979-fb5jh\" (UID: \"33f30bb4-e89e-4c99-9801-787321b0a13d\") " pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:34:15.473776 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.473650    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33f30bb4-e89e-4c99-9801-787321b0a13d-tmp-dir\") pod \"auth-test-model-751f6a-kserve-888dfd979-fb5jh\" (UID: \"33f30bb4-e89e-4c99-9801-787321b0a13d\") " pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:34:15.473776 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.473667    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33f30bb4-e89e-4c99-9801-787321b0a13d-home\") pod \"auth-test-model-751f6a-kserve-888dfd979-fb5jh\" (UID: \"33f30bb4-e89e-4c99-9801-787321b0a13d\") " pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:34:15.473776 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.473698    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33f30bb4-e89e-4c99-9801-787321b0a13d-tls-certs\") pod \"auth-test-model-751f6a-kserve-888dfd979-fb5jh\" (UID: \"33f30bb4-e89e-4c99-9801-787321b0a13d\") " pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:34:15.474013 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.473842    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68szp\" (UniqueName: \"kubernetes.io/projected/33f30bb4-e89e-4c99-9801-787321b0a13d-kube-api-access-68szp\") pod \"auth-test-model-751f6a-kserve-888dfd979-fb5jh\" (UID: \"33f30bb4-e89e-4c99-9801-787321b0a13d\") " pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:34:15.474073 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.474035    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33f30bb4-e89e-4c99-9801-787321b0a13d-model-cache\") pod \"auth-test-model-751f6a-kserve-888dfd979-fb5jh\" (UID: \"33f30bb4-e89e-4c99-9801-787321b0a13d\") " pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:34:15.474073 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.474060    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33f30bb4-e89e-4c99-9801-787321b0a13d-tmp-dir\") pod \"auth-test-model-751f6a-kserve-888dfd979-fb5jh\" (UID: \"33f30bb4-e89e-4c99-9801-787321b0a13d\") " pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:34:15.474167 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.474103    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33f30bb4-e89e-4c99-9801-787321b0a13d-home\") pod \"auth-test-model-751f6a-kserve-888dfd979-fb5jh\" (UID: \"33f30bb4-e89e-4c99-9801-787321b0a13d\") " pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:34:15.476068 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.476049    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33f30bb4-e89e-4c99-9801-787321b0a13d-dshm\") pod \"auth-test-model-751f6a-kserve-888dfd979-fb5jh\" (UID: \"33f30bb4-e89e-4c99-9801-787321b0a13d\") " pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:34:15.476322 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.476306    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33f30bb4-e89e-4c99-9801-787321b0a13d-tls-certs\") pod \"auth-test-model-751f6a-kserve-888dfd979-fb5jh\" (UID: \"33f30bb4-e89e-4c99-9801-787321b0a13d\") " pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:34:15.482785 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.482765    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68szp\" (UniqueName: \"kubernetes.io/projected/33f30bb4-e89e-4c99-9801-787321b0a13d-kube-api-access-68szp\") pod \"auth-test-model-751f6a-kserve-888dfd979-fb5jh\" (UID: \"33f30bb4-e89e-4c99-9801-787321b0a13d\") " pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:34:15.559100 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.559019    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:34:15.834154 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:15.834126    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"]
Sep 19 01:34:15.836716 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:34:15.836689    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33f30bb4_e89e_4c99_9801_787321b0a13d.slice/crio-faa465477c7602de863860b150f7d184bc6c89a23a11d10911cdaf7e40215928 WatchSource:0}: Error finding container faa465477c7602de863860b150f7d184bc6c89a23a11d10911cdaf7e40215928: Status 404 returned error can't find the container with id faa465477c7602de863860b150f7d184bc6c89a23a11d10911cdaf7e40215928
Sep 19 01:34:16.351925 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:16.351883    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh" event={"ID":"33f30bb4-e89e-4c99-9801-787321b0a13d","Type":"ContainerStarted","Data":"faa465477c7602de863860b150f7d184bc6c89a23a11d10911cdaf7e40215928"}
Sep 19 01:34:16.945991 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:16.945960    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"]
Sep 19 01:34:16.946213 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:16.946165    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7" podUID="70fbeffe-5bd4-4821-a7c6-e596118f862e" containerName="maas-api" containerID="cri-o://c2aab50b90016652dd6cbafcd7a707651bb81363437ca3c3fe0f485c9636fbc5" gracePeriod=30
Sep 19 01:34:17.210598 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:17.210547    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"
Sep 19 01:34:17.360946 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:17.360913    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh" event={"ID":"33f30bb4-e89e-4c99-9801-787321b0a13d","Type":"ContainerStarted","Data":"c796809a33f774030333ee911139e1fb44056feada10aecb77cada87575610b7"}
Sep 19 01:34:17.362329 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:17.362303    2580 generic.go:356] "Generic (PLEG): container finished" podID="70fbeffe-5bd4-4821-a7c6-e596118f862e" containerID="c2aab50b90016652dd6cbafcd7a707651bb81363437ca3c3fe0f485c9636fbc5" exitCode=0
Sep 19 01:34:17.362447 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:17.362359    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"
Sep 19 01:34:17.362447 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:17.362387    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7" event={"ID":"70fbeffe-5bd4-4821-a7c6-e596118f862e","Type":"ContainerDied","Data":"c2aab50b90016652dd6cbafcd7a707651bb81363437ca3c3fe0f485c9636fbc5"}
Sep 19 01:34:17.362447 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:17.362433    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7" event={"ID":"70fbeffe-5bd4-4821-a7c6-e596118f862e","Type":"ContainerDied","Data":"abb8fde56a1942859e9e761398ea21719c9a2854ebad6a9cce54d61e6a0fc097"}
Sep 19 01:34:17.362585 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:17.362454    2580 scope.go:117] "RemoveContainer" containerID="c2aab50b90016652dd6cbafcd7a707651bb81363437ca3c3fe0f485c9636fbc5"
Sep 19 01:34:17.372199 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:17.372184    2580 scope.go:117] "RemoveContainer" containerID="c2aab50b90016652dd6cbafcd7a707651bb81363437ca3c3fe0f485c9636fbc5"
Sep 19 01:34:17.372447 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:34:17.372429    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2aab50b90016652dd6cbafcd7a707651bb81363437ca3c3fe0f485c9636fbc5\": container with ID starting with c2aab50b90016652dd6cbafcd7a707651bb81363437ca3c3fe0f485c9636fbc5 not found: ID does not exist" containerID="c2aab50b90016652dd6cbafcd7a707651bb81363437ca3c3fe0f485c9636fbc5"
Sep 19 01:34:17.372534 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:17.372459    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2aab50b90016652dd6cbafcd7a707651bb81363437ca3c3fe0f485c9636fbc5"} err="failed to get container status \"c2aab50b90016652dd6cbafcd7a707651bb81363437ca3c3fe0f485c9636fbc5\": rpc error: code = NotFound desc = could not find container \"c2aab50b90016652dd6cbafcd7a707651bb81363437ca3c3fe0f485c9636fbc5\": container with ID starting with c2aab50b90016652dd6cbafcd7a707651bb81363437ca3c3fe0f485c9636fbc5 not found: ID does not exist"
Sep 19 01:34:17.388174 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:17.388127    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh" podStartSLOduration=1.262457887 podStartE2EDuration="2.388112006s" totalImagesPullingTime="1.125654119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:34:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:34:17.386741137 +0000 UTC m=+1312.293603184" watchObservedRunningTime="2026-09-19 01:34:17.388112006 +0000 UTC m=+1312.294974055"
Sep 19 01:34:17.394874 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:17.394851    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70fbeffe-5bd4-4821-a7c6-e596118f862e-metrics-certs\") pod \"70fbeffe-5bd4-4821-a7c6-e596118f862e\" (UID: \"70fbeffe-5bd4-4821-a7c6-e596118f862e\") "
Sep 19 01:34:17.395065 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:17.395046    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsz4z\" (UniqueName: \"kubernetes.io/projected/70fbeffe-5bd4-4821-a7c6-e596118f862e-kube-api-access-lsz4z\") pod \"70fbeffe-5bd4-4821-a7c6-e596118f862e\" (UID: \"70fbeffe-5bd4-4821-a7c6-e596118f862e\") "
Sep 19 01:34:17.395137 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:17.395115    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70fbeffe-5bd4-4821-a7c6-e596118f862e-openshift-service-ca\") pod \"70fbeffe-5bd4-4821-a7c6-e596118f862e\" (UID: \"70fbeffe-5bd4-4821-a7c6-e596118f862e\") "
Sep 19 01:34:17.395207 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:17.395191    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70fbeffe-5bd4-4821-a7c6-e596118f862e-maas-api-tls\") pod \"70fbeffe-5bd4-4821-a7c6-e596118f862e\" (UID: \"70fbeffe-5bd4-4821-a7c6-e596118f862e\") "
Sep 19 01:34:17.395602 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:17.395571    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70fbeffe-5bd4-4821-a7c6-e596118f862e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "70fbeffe-5bd4-4821-a7c6-e596118f862e" (UID: "70fbeffe-5bd4-4821-a7c6-e596118f862e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:34:17.397121 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:17.397099    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70fbeffe-5bd4-4821-a7c6-e596118f862e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "70fbeffe-5bd4-4821-a7c6-e596118f862e" (UID: "70fbeffe-5bd4-4821-a7c6-e596118f862e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:34:17.397272 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:17.397249    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70fbeffe-5bd4-4821-a7c6-e596118f862e-kube-api-access-lsz4z" (OuterVolumeSpecName: "kube-api-access-lsz4z") pod "70fbeffe-5bd4-4821-a7c6-e596118f862e" (UID: "70fbeffe-5bd4-4821-a7c6-e596118f862e"). InnerVolumeSpecName "kube-api-access-lsz4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:34:17.397272 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:17.397265    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70fbeffe-5bd4-4821-a7c6-e596118f862e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "70fbeffe-5bd4-4821-a7c6-e596118f862e" (UID: "70fbeffe-5bd4-4821-a7c6-e596118f862e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:34:17.496176 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:17.496141    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70fbeffe-5bd4-4821-a7c6-e596118f862e-openshift-service-ca\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:34:17.496176 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:17.496176    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70fbeffe-5bd4-4821-a7c6-e596118f862e-maas-api-tls\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:34:17.496375 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:17.496189    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70fbeffe-5bd4-4821-a7c6-e596118f862e-metrics-certs\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:34:17.496375 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:17.496203    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsz4z\" (UniqueName: \"kubernetes.io/projected/70fbeffe-5bd4-4821-a7c6-e596118f862e-kube-api-access-lsz4z\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:34:17.696817 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:17.696783    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"]
Sep 19 01:34:17.706579 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:17.706550    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b5fe4519-6f8bc6fc69-xrjs7"]
Sep 19 01:34:19.066005 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:19.065968    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70fbeffe-5bd4-4821-a7c6-e596118f862e" path="/var/lib/kubelet/pods/70fbeffe-5bd4-4821-a7c6-e596118f862e/volumes"
Sep 19 01:34:25.560222 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:25.560180    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:34:25.560646 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:25.560246    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:34:25.572614 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:25.572593    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:34:26.437355 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:26.437323    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:34:54.785471 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:54.785381    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"]
Sep 19 01:34:54.786129 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:54.786109    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70fbeffe-5bd4-4821-a7c6-e596118f862e" containerName="maas-api"
Sep 19 01:34:54.786214 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:54.786130    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="70fbeffe-5bd4-4821-a7c6-e596118f862e" containerName="maas-api"
Sep 19 01:34:54.786294 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:54.786280    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="70fbeffe-5bd4-4821-a7c6-e596118f862e" containerName="maas-api"
Sep 19 01:34:54.790849 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:54.790827    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:34:54.793199 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:54.793177    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-bebef4\"/\"kube-root-ca.crt\""
Sep 19 01:34:54.793314 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:54.793289    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-bebef4\"/\"openshift-service-ca.crt\""
Sep 19 01:34:54.793886 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:54.793867    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-bebef4\"/\"default-dockercfg-lvlgd\""
Sep 19 01:34:54.794001 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:54.793923    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-bebef4\"/\"auth-test-model-bebef4-kserve-self-signed-certs\""
Sep 19 01:34:54.800919 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:54.800887    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"]
Sep 19 01:34:54.951949 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:54.951915    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c7bac26-42b5-4b3d-9a03-8205d2691f96-dshm\") pod \"auth-test-model-bebef4-kserve-f7474ccd7-qgmps\" (UID: \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\") " pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:34:54.951949 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:54.951946    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c7bac26-42b5-4b3d-9a03-8205d2691f96-tls-certs\") pod \"auth-test-model-bebef4-kserve-f7474ccd7-qgmps\" (UID: \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\") " pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:34:54.952211 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:54.951974    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c7bac26-42b5-4b3d-9a03-8205d2691f96-tmp-dir\") pod \"auth-test-model-bebef4-kserve-f7474ccd7-qgmps\" (UID: \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\") " pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:34:54.952211 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:54.951994    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c7bac26-42b5-4b3d-9a03-8205d2691f96-home\") pod \"auth-test-model-bebef4-kserve-f7474ccd7-qgmps\" (UID: \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\") " pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:34:54.952211 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:54.952075    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvlhq\" (UniqueName: \"kubernetes.io/projected/2c7bac26-42b5-4b3d-9a03-8205d2691f96-kube-api-access-pvlhq\") pod \"auth-test-model-bebef4-kserve-f7474ccd7-qgmps\" (UID: \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\") " pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:34:54.952211 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:54.952111    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c7bac26-42b5-4b3d-9a03-8205d2691f96-model-cache\") pod \"auth-test-model-bebef4-kserve-f7474ccd7-qgmps\" (UID: \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\") " pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:34:55.053033 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:55.052951    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c7bac26-42b5-4b3d-9a03-8205d2691f96-dshm\") pod \"auth-test-model-bebef4-kserve-f7474ccd7-qgmps\" (UID: \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\") " pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:34:55.053033 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:55.052985    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c7bac26-42b5-4b3d-9a03-8205d2691f96-tls-certs\") pod \"auth-test-model-bebef4-kserve-f7474ccd7-qgmps\" (UID: \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\") " pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:34:55.053033 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:55.053008    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c7bac26-42b5-4b3d-9a03-8205d2691f96-tmp-dir\") pod \"auth-test-model-bebef4-kserve-f7474ccd7-qgmps\" (UID: \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\") " pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:34:55.053033 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:55.053032    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c7bac26-42b5-4b3d-9a03-8205d2691f96-home\") pod \"auth-test-model-bebef4-kserve-f7474ccd7-qgmps\" (UID: \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\") " pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:34:55.053290 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:55.053138    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvlhq\" (UniqueName: \"kubernetes.io/projected/2c7bac26-42b5-4b3d-9a03-8205d2691f96-kube-api-access-pvlhq\") pod \"auth-test-model-bebef4-kserve-f7474ccd7-qgmps\" (UID: \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\") " pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:34:55.053290 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:55.053181    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c7bac26-42b5-4b3d-9a03-8205d2691f96-model-cache\") pod \"auth-test-model-bebef4-kserve-f7474ccd7-qgmps\" (UID: \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\") " pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:34:55.053440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:55.053420    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c7bac26-42b5-4b3d-9a03-8205d2691f96-tmp-dir\") pod \"auth-test-model-bebef4-kserve-f7474ccd7-qgmps\" (UID: \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\") " pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:34:55.053534 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:55.053437    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c7bac26-42b5-4b3d-9a03-8205d2691f96-home\") pod \"auth-test-model-bebef4-kserve-f7474ccd7-qgmps\" (UID: \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\") " pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:34:55.053654 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:55.053633    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c7bac26-42b5-4b3d-9a03-8205d2691f96-model-cache\") pod \"auth-test-model-bebef4-kserve-f7474ccd7-qgmps\" (UID: \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\") " pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:34:55.055225 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:55.055205    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c7bac26-42b5-4b3d-9a03-8205d2691f96-dshm\") pod \"auth-test-model-bebef4-kserve-f7474ccd7-qgmps\" (UID: \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\") " pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:34:55.055539 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:55.055521    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c7bac26-42b5-4b3d-9a03-8205d2691f96-tls-certs\") pod \"auth-test-model-bebef4-kserve-f7474ccd7-qgmps\" (UID: \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\") " pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:34:55.063844 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:55.063822    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvlhq\" (UniqueName: \"kubernetes.io/projected/2c7bac26-42b5-4b3d-9a03-8205d2691f96-kube-api-access-pvlhq\") pod \"auth-test-model-bebef4-kserve-f7474ccd7-qgmps\" (UID: \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\") " pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:34:55.103266 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:55.103245    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:34:55.383100 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:55.380819    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"]
Sep 19 01:34:55.385197 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:34:55.385169    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c7bac26_42b5_4b3d_9a03_8205d2691f96.slice/crio-cb6afaa82cae1e577a90e45d6bdbb0ec1c9674077992114b066caafea3b79aa6 WatchSource:0}: Error finding container cb6afaa82cae1e577a90e45d6bdbb0ec1c9674077992114b066caafea3b79aa6: Status 404 returned error can't find the container with id cb6afaa82cae1e577a90e45d6bdbb0ec1c9674077992114b066caafea3b79aa6
Sep 19 01:34:55.615623 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:55.615590    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps" event={"ID":"2c7bac26-42b5-4b3d-9a03-8205d2691f96","Type":"ContainerStarted","Data":"035a7edb69d2120f03e368fe6d571586f6cc56e3080208ac40acf1904d0909ae"}
Sep 19 01:34:55.615813 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:55.615637    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps" event={"ID":"2c7bac26-42b5-4b3d-9a03-8205d2691f96","Type":"ContainerStarted","Data":"cb6afaa82cae1e577a90e45d6bdbb0ec1c9674077992114b066caafea3b79aa6"}
Sep 19 01:34:55.634008 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:34:55.633917    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps" podStartSLOduration=1.63390189 podStartE2EDuration="1.63390189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:34:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:34:55.63228341 +0000 UTC m=+1350.539145471" watchObservedRunningTime="2026-09-19 01:34:55.63390189 +0000 UTC m=+1350.540764006"
Sep 19 01:35:05.104332 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:35:05.104295    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:35:05.104810 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:35:05.104344    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:35:05.117299 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:35:05.117280    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:35:05.693612 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:35:05.693580    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:36:27.337264 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:27.337189    2580 scope.go:117] "RemoveContainer" containerID="a64ef06986125f46e6fc75a82e4731a7d2e60aa834556006b4cd645c99a1ab28"
Sep 19 01:36:27.735835 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:27.735806    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"]
Sep 19 01:36:27.736050 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:27.736007    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm" podUID="7d5a338f-fcf2-4b78-8f61-81326a40b7a9" containerName="maas-api" containerID="cri-o://8cab26403ad4a938a1733bfccb1b615238937aeb5e3d9fb66622b2ee81863043" gracePeriod=30
Sep 19 01:36:27.799950 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:27.799901    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz"]
Sep 19 01:36:27.800797 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:27.800753    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz" podUID="d1bff2cc-199c-47c3-aacd-db6b0f522a91" containerName="payload-processing" containerID="cri-o://86bc4c12df4f5bd60f772f27078d6243fea08f5b6c67cc785ea90f740b60b712" gracePeriod=30
Sep 19 01:36:28.080369 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.080347    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"
Sep 19 01:36:28.096198 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.096176    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz"
Sep 19 01:36:28.179727 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.179691    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-metrics-certs\") pod \"7d5a338f-fcf2-4b78-8f61-81326a40b7a9\" (UID: \"7d5a338f-fcf2-4b78-8f61-81326a40b7a9\") "
Sep 19 01:36:28.179727 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.179733    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1bff2cc-199c-47c3-aacd-db6b0f522a91-plugins-config-volume\") pod \"d1bff2cc-199c-47c3-aacd-db6b0f522a91\" (UID: \"d1bff2cc-199c-47c3-aacd-db6b0f522a91\") "
Sep 19 01:36:28.179939 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.179852    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-openshift-service-ca\") pod \"7d5a338f-fcf2-4b78-8f61-81326a40b7a9\" (UID: \"7d5a338f-fcf2-4b78-8f61-81326a40b7a9\") "
Sep 19 01:36:28.179939 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.179892    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z525t\" (UniqueName: \"kubernetes.io/projected/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-kube-api-access-z525t\") pod \"7d5a338f-fcf2-4b78-8f61-81326a40b7a9\" (UID: \"7d5a338f-fcf2-4b78-8f61-81326a40b7a9\") "
Sep 19 01:36:28.179939 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.179933    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-maas-api-tls\") pod \"7d5a338f-fcf2-4b78-8f61-81326a40b7a9\" (UID: \"7d5a338f-fcf2-4b78-8f61-81326a40b7a9\") "
Sep 19 01:36:28.180104 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.179968    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rgh7j\" (UniqueName: \"kubernetes.io/projected/d1bff2cc-199c-47c3-aacd-db6b0f522a91-kube-api-access-rgh7j\") pod \"d1bff2cc-199c-47c3-aacd-db6b0f522a91\" (UID: \"d1bff2cc-199c-47c3-aacd-db6b0f522a91\") "
Sep 19 01:36:28.180224 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.180189    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1bff2cc-199c-47c3-aacd-db6b0f522a91-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d1bff2cc-199c-47c3-aacd-db6b0f522a91" (UID: "d1bff2cc-199c-47c3-aacd-db6b0f522a91"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:36:28.180328 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.180259    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7d5a338f-fcf2-4b78-8f61-81326a40b7a9" (UID: "7d5a338f-fcf2-4b78-8f61-81326a40b7a9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:36:28.180741 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.180718    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1bff2cc-199c-47c3-aacd-db6b0f522a91-plugins-config-volume\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:36:28.180891 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.180876    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-openshift-service-ca\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:36:28.182128 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.182094    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7d5a338f-fcf2-4b78-8f61-81326a40b7a9" (UID: "7d5a338f-fcf2-4b78-8f61-81326a40b7a9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:36:28.182232 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.182201    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7d5a338f-fcf2-4b78-8f61-81326a40b7a9" (UID: "7d5a338f-fcf2-4b78-8f61-81326a40b7a9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:36:28.182289 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.182265    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1bff2cc-199c-47c3-aacd-db6b0f522a91-kube-api-access-rgh7j" (OuterVolumeSpecName: "kube-api-access-rgh7j") pod "d1bff2cc-199c-47c3-aacd-db6b0f522a91" (UID: "d1bff2cc-199c-47c3-aacd-db6b0f522a91"). InnerVolumeSpecName "kube-api-access-rgh7j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:36:28.182360 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.182341    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-kube-api-access-z525t" (OuterVolumeSpecName: "kube-api-access-z525t") pod "7d5a338f-fcf2-4b78-8f61-81326a40b7a9" (UID: "7d5a338f-fcf2-4b78-8f61-81326a40b7a9"). InnerVolumeSpecName "kube-api-access-z525t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:36:28.217402 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.217373    2580 generic.go:356] "Generic (PLEG): container finished" podID="d1bff2cc-199c-47c3-aacd-db6b0f522a91" containerID="86bc4c12df4f5bd60f772f27078d6243fea08f5b6c67cc785ea90f740b60b712" exitCode=0
Sep 19 01:36:28.217532 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.217439    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz"
Sep 19 01:36:28.217532 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.217459    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz" event={"ID":"d1bff2cc-199c-47c3-aacd-db6b0f522a91","Type":"ContainerDied","Data":"86bc4c12df4f5bd60f772f27078d6243fea08f5b6c67cc785ea90f740b60b712"}
Sep 19 01:36:28.217656 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.217571    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz" event={"ID":"d1bff2cc-199c-47c3-aacd-db6b0f522a91","Type":"ContainerDied","Data":"ceaf564285760da230853da457f7d4b53af5f8adbf6b0ed5720e17fa387e11af"}
Sep 19 01:36:28.217656 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.217609    2580 scope.go:117] "RemoveContainer" containerID="86bc4c12df4f5bd60f772f27078d6243fea08f5b6c67cc785ea90f740b60b712"
Sep 19 01:36:28.218973 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.218949    2580 generic.go:356] "Generic (PLEG): container finished" podID="7d5a338f-fcf2-4b78-8f61-81326a40b7a9" containerID="8cab26403ad4a938a1733bfccb1b615238937aeb5e3d9fb66622b2ee81863043" exitCode=0
Sep 19 01:36:28.219053 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.218986    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm" event={"ID":"7d5a338f-fcf2-4b78-8f61-81326a40b7a9","Type":"ContainerDied","Data":"8cab26403ad4a938a1733bfccb1b615238937aeb5e3d9fb66622b2ee81863043"}
Sep 19 01:36:28.219053 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.219000    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"
Sep 19 01:36:28.219053 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.219022    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm" event={"ID":"7d5a338f-fcf2-4b78-8f61-81326a40b7a9","Type":"ContainerDied","Data":"7d74f9c983f22a469675213412b856d3858513eab9f8a035ce7e3a1d83a7a58e"}
Sep 19 01:36:28.229304 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.229213    2580 scope.go:117] "RemoveContainer" containerID="86bc4c12df4f5bd60f772f27078d6243fea08f5b6c67cc785ea90f740b60b712"
Sep 19 01:36:28.229616 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:36:28.229591    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86bc4c12df4f5bd60f772f27078d6243fea08f5b6c67cc785ea90f740b60b712\": container with ID starting with 86bc4c12df4f5bd60f772f27078d6243fea08f5b6c67cc785ea90f740b60b712 not found: ID does not exist" containerID="86bc4c12df4f5bd60f772f27078d6243fea08f5b6c67cc785ea90f740b60b712"
Sep 19 01:36:28.229800 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.229767    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86bc4c12df4f5bd60f772f27078d6243fea08f5b6c67cc785ea90f740b60b712"} err="failed to get container status \"86bc4c12df4f5bd60f772f27078d6243fea08f5b6c67cc785ea90f740b60b712\": rpc error: code = NotFound desc = could not find container \"86bc4c12df4f5bd60f772f27078d6243fea08f5b6c67cc785ea90f740b60b712\": container with ID starting with 86bc4c12df4f5bd60f772f27078d6243fea08f5b6c67cc785ea90f740b60b712 not found: ID does not exist"
Sep 19 01:36:28.229902 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.229810    2580 scope.go:117] "RemoveContainer" containerID="8cab26403ad4a938a1733bfccb1b615238937aeb5e3d9fb66622b2ee81863043"
Sep 19 01:36:28.239213 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.239196    2580 scope.go:117] "RemoveContainer" containerID="8cab26403ad4a938a1733bfccb1b615238937aeb5e3d9fb66622b2ee81863043"
Sep 19 01:36:28.239547 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:36:28.239518    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cab26403ad4a938a1733bfccb1b615238937aeb5e3d9fb66622b2ee81863043\": container with ID starting with 8cab26403ad4a938a1733bfccb1b615238937aeb5e3d9fb66622b2ee81863043 not found: ID does not exist" containerID="8cab26403ad4a938a1733bfccb1b615238937aeb5e3d9fb66622b2ee81863043"
Sep 19 01:36:28.239615 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.239553    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cab26403ad4a938a1733bfccb1b615238937aeb5e3d9fb66622b2ee81863043"} err="failed to get container status \"8cab26403ad4a938a1733bfccb1b615238937aeb5e3d9fb66622b2ee81863043\": rpc error: code = NotFound desc = could not find container \"8cab26403ad4a938a1733bfccb1b615238937aeb5e3d9fb66622b2ee81863043\": container with ID starting with 8cab26403ad4a938a1733bfccb1b615238937aeb5e3d9fb66622b2ee81863043 not found: ID does not exist"
Sep 19 01:36:28.248685 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.248624    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz"]
Sep 19 01:36:28.251431 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.251407    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-8e708f-55b689fd47-kv4kz"]
Sep 19 01:36:28.260111 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.260082    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"]
Sep 19 01:36:28.264233 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.264183    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-8e708f-7566c7cc9f-skgmm"]
Sep 19 01:36:28.282132 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.282111    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgh7j\" (UniqueName: \"kubernetes.io/projected/d1bff2cc-199c-47c3-aacd-db6b0f522a91-kube-api-access-rgh7j\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:36:28.282132 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.282133    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-metrics-certs\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:36:28.282301 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.282144    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z525t\" (UniqueName: \"kubernetes.io/projected/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-kube-api-access-z525t\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:36:28.282301 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:28.282152    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d5a338f-fcf2-4b78-8f61-81326a40b7a9-maas-api-tls\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:36:29.064788 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:29.064709    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d5a338f-fcf2-4b78-8f61-81326a40b7a9" path="/var/lib/kubelet/pods/7d5a338f-fcf2-4b78-8f61-81326a40b7a9/volumes"
Sep 19 01:36:29.065132 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:29.065107    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1bff2cc-199c-47c3-aacd-db6b0f522a91" path="/var/lib/kubelet/pods/d1bff2cc-199c-47c3-aacd-db6b0f522a91/volumes"
Sep 19 01:36:38.389144 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.389115    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw"]
Sep 19 01:36:38.389820 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.389801    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d5a338f-fcf2-4b78-8f61-81326a40b7a9" containerName="maas-api"
Sep 19 01:36:38.389913 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.389823    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d5a338f-fcf2-4b78-8f61-81326a40b7a9" containerName="maas-api"
Sep 19 01:36:38.389913 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.389841    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1bff2cc-199c-47c3-aacd-db6b0f522a91" containerName="payload-processing"
Sep 19 01:36:38.389913 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.389850    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1bff2cc-199c-47c3-aacd-db6b0f522a91" containerName="payload-processing"
Sep 19 01:36:38.390066 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.389982    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d5a338f-fcf2-4b78-8f61-81326a40b7a9" containerName="maas-api"
Sep 19 01:36:38.390066 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.389998    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1bff2cc-199c-47c3-aacd-db6b0f522a91" containerName="payload-processing"
Sep 19 01:36:38.394299 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.394273    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw"
Sep 19 01:36:38.396515 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.396481    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-a7b600-serving-cert\""
Sep 19 01:36:38.406083 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.406058    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw"]
Sep 19 01:36:38.490297 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.490265    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7be56cd0-b212-4565-b84f-4d994a978624-maas-api-tls\") pod \"maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw\" (UID: \"7be56cd0-b212-4565-b84f-4d994a978624\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw"
Sep 19 01:36:38.490508 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.490436    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7be56cd0-b212-4565-b84f-4d994a978624-metrics-certs\") pod \"maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw\" (UID: \"7be56cd0-b212-4565-b84f-4d994a978624\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw"
Sep 19 01:36:38.490631 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.490534    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfznt\" (UniqueName: \"kubernetes.io/projected/7be56cd0-b212-4565-b84f-4d994a978624-kube-api-access-nfznt\") pod \"maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw\" (UID: \"7be56cd0-b212-4565-b84f-4d994a978624\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw"
Sep 19 01:36:38.490631 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.490573    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7be56cd0-b212-4565-b84f-4d994a978624-openshift-service-ca\") pod \"maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw\" (UID: \"7be56cd0-b212-4565-b84f-4d994a978624\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw"
Sep 19 01:36:38.591430 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.591397    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7be56cd0-b212-4565-b84f-4d994a978624-maas-api-tls\") pod \"maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw\" (UID: \"7be56cd0-b212-4565-b84f-4d994a978624\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw"
Sep 19 01:36:38.591635 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.591462    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7be56cd0-b212-4565-b84f-4d994a978624-metrics-certs\") pod \"maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw\" (UID: \"7be56cd0-b212-4565-b84f-4d994a978624\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw"
Sep 19 01:36:38.591635 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.591523    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfznt\" (UniqueName: \"kubernetes.io/projected/7be56cd0-b212-4565-b84f-4d994a978624-kube-api-access-nfznt\") pod \"maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw\" (UID: \"7be56cd0-b212-4565-b84f-4d994a978624\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw"
Sep 19 01:36:38.591635 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.591545    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7be56cd0-b212-4565-b84f-4d994a978624-openshift-service-ca\") pod \"maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw\" (UID: \"7be56cd0-b212-4565-b84f-4d994a978624\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw"
Sep 19 01:36:38.592187 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.592167    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7be56cd0-b212-4565-b84f-4d994a978624-openshift-service-ca\") pod \"maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw\" (UID: \"7be56cd0-b212-4565-b84f-4d994a978624\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw"
Sep 19 01:36:38.593985 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.593964    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7be56cd0-b212-4565-b84f-4d994a978624-metrics-certs\") pod \"maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw\" (UID: \"7be56cd0-b212-4565-b84f-4d994a978624\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw"
Sep 19 01:36:38.593985 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.593978    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7be56cd0-b212-4565-b84f-4d994a978624-maas-api-tls\") pod \"maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw\" (UID: \"7be56cd0-b212-4565-b84f-4d994a978624\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw"
Sep 19 01:36:38.600177 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.600149    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfznt\" (UniqueName: \"kubernetes.io/projected/7be56cd0-b212-4565-b84f-4d994a978624-kube-api-access-nfznt\") pod \"maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw\" (UID: \"7be56cd0-b212-4565-b84f-4d994a978624\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw"
Sep 19 01:36:38.708996 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.708913    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw"
Sep 19 01:36:38.995435 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.995405    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw"]
Sep 19 01:36:38.996180 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:36:38.996151    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7be56cd0_b212_4565_b84f_4d994a978624.slice/crio-d0f5af8db7079a2951db9587eadff9bf3669a4d69a570fb0e7f4b502bfa04685 WatchSource:0}: Error finding container d0f5af8db7079a2951db9587eadff9bf3669a4d69a570fb0e7f4b502bfa04685: Status 404 returned error can't find the container with id d0f5af8db7079a2951db9587eadff9bf3669a4d69a570fb0e7f4b502bfa04685
Sep 19 01:36:38.997565 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:38.997550    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 01:36:39.296706 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:39.296624    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw" event={"ID":"7be56cd0-b212-4565-b84f-4d994a978624","Type":"ContainerStarted","Data":"d0f5af8db7079a2951db9587eadff9bf3669a4d69a570fb0e7f4b502bfa04685"}
Sep 19 01:36:40.305107 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:40.305073    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw" event={"ID":"7be56cd0-b212-4565-b84f-4d994a978624","Type":"ContainerStarted","Data":"481cc039210fe9350e0946677f2fd53d17943ad4dd9f39dcb983db461a3a353c"}
Sep 19 01:36:40.305518 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:40.305183    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw"
Sep 19 01:36:40.324471 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:40.324423    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw" podStartSLOduration=1.923817447 podStartE2EDuration="2.324407813s" totalImagesPullingTime="400.590366ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:36:40.322778705 +0000 UTC m=+1455.229640754" watchObservedRunningTime="2026-09-19 01:36:40.324407813 +0000 UTC m=+1455.231269861"
Sep 19 01:36:46.317690 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:36:46.317657    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw"
Sep 19 01:37:13.784912 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:13.784880    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7"]
Sep 19 01:37:13.789305 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:13.789287    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7"
Sep 19 01:37:13.797650 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:13.797099    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-29d76c-serving-cert\""
Sep 19 01:37:13.809204 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:13.809182    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7"]
Sep 19 01:37:13.948464 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:13.948429    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slfwz\" (UniqueName: \"kubernetes.io/projected/c834b565-3fe9-4da0-8f6f-608c0f015c5a-kube-api-access-slfwz\") pod \"maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7\" (UID: \"c834b565-3fe9-4da0-8f6f-608c0f015c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7"
Sep 19 01:37:13.948464 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:13.948470    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c834b565-3fe9-4da0-8f6f-608c0f015c5a-maas-api-tls\") pod \"maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7\" (UID: \"c834b565-3fe9-4da0-8f6f-608c0f015c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7"
Sep 19 01:37:13.948720 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:13.948486    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c834b565-3fe9-4da0-8f6f-608c0f015c5a-metrics-certs\") pod \"maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7\" (UID: \"c834b565-3fe9-4da0-8f6f-608c0f015c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7"
Sep 19 01:37:13.948720 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:13.948609    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c834b565-3fe9-4da0-8f6f-608c0f015c5a-openshift-service-ca\") pod \"maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7\" (UID: \"c834b565-3fe9-4da0-8f6f-608c0f015c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7"
Sep 19 01:37:14.049687 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:14.049607    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slfwz\" (UniqueName: \"kubernetes.io/projected/c834b565-3fe9-4da0-8f6f-608c0f015c5a-kube-api-access-slfwz\") pod \"maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7\" (UID: \"c834b565-3fe9-4da0-8f6f-608c0f015c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7"
Sep 19 01:37:14.049687 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:14.049649    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c834b565-3fe9-4da0-8f6f-608c0f015c5a-maas-api-tls\") pod \"maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7\" (UID: \"c834b565-3fe9-4da0-8f6f-608c0f015c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7"
Sep 19 01:37:14.049687 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:14.049665    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c834b565-3fe9-4da0-8f6f-608c0f015c5a-metrics-certs\") pod \"maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7\" (UID: \"c834b565-3fe9-4da0-8f6f-608c0f015c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7"
Sep 19 01:37:14.049963 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:14.049700    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c834b565-3fe9-4da0-8f6f-608c0f015c5a-openshift-service-ca\") pod \"maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7\" (UID: \"c834b565-3fe9-4da0-8f6f-608c0f015c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7"
Sep 19 01:37:14.050422 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:14.050398    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c834b565-3fe9-4da0-8f6f-608c0f015c5a-openshift-service-ca\") pod \"maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7\" (UID: \"c834b565-3fe9-4da0-8f6f-608c0f015c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7"
Sep 19 01:37:14.052144 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:14.052114    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c834b565-3fe9-4da0-8f6f-608c0f015c5a-maas-api-tls\") pod \"maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7\" (UID: \"c834b565-3fe9-4da0-8f6f-608c0f015c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7"
Sep 19 01:37:14.052144 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:14.052138    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c834b565-3fe9-4da0-8f6f-608c0f015c5a-metrics-certs\") pod \"maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7\" (UID: \"c834b565-3fe9-4da0-8f6f-608c0f015c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7"
Sep 19 01:37:14.059867 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:14.059845    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slfwz\" (UniqueName: \"kubernetes.io/projected/c834b565-3fe9-4da0-8f6f-608c0f015c5a-kube-api-access-slfwz\") pod \"maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7\" (UID: \"c834b565-3fe9-4da0-8f6f-608c0f015c5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7"
Sep 19 01:37:14.101364 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:14.101325    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7"
Sep 19 01:37:14.379882 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:14.379852    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7"]
Sep 19 01:37:14.381943 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:37:14.381916    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc834b565_3fe9_4da0_8f6f_608c0f015c5a.slice/crio-d38d8920fe703f0597fd3c1a34c465937f4b72417fd29a7362676132963beb8d WatchSource:0}: Error finding container d38d8920fe703f0597fd3c1a34c465937f4b72417fd29a7362676132963beb8d: Status 404 returned error can't find the container with id d38d8920fe703f0597fd3c1a34c465937f4b72417fd29a7362676132963beb8d
Sep 19 01:37:14.546835 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:14.546796    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7" event={"ID":"c834b565-3fe9-4da0-8f6f-608c0f015c5a","Type":"ContainerStarted","Data":"d38d8920fe703f0597fd3c1a34c465937f4b72417fd29a7362676132963beb8d"}
Sep 19 01:37:15.556093 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:15.556053    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7" event={"ID":"c834b565-3fe9-4da0-8f6f-608c0f015c5a","Type":"ContainerStarted","Data":"ee2d0628faa699ca27d7523d737704b905109a90ca62d607fc2de3a2b03b89b0"}
Sep 19 01:37:15.556513 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:15.556239    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7"
Sep 19 01:37:15.578270 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:15.578222    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7" podStartSLOduration=2.258317556 podStartE2EDuration="2.578206615s" totalImagesPullingTime="319.889059ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:37:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:37:15.574775569 +0000 UTC m=+1490.481637617" watchObservedRunningTime="2026-09-19 01:37:15.578206615 +0000 UTC m=+1490.485068679"
Sep 19 01:37:21.567872 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:21.567839    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7"
Sep 19 01:37:27.214241 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:27.214212    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f6bdh_74c8e2f6-1b01-4bd1-b491-ad407a15a109/console-operator/1.log"
Sep 19 01:37:27.219236 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:27.219212    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f6bdh_74c8e2f6-1b01-4bd1-b491-ad407a15a109/console-operator/1.log"
Sep 19 01:37:49.649125 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:49.649057    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7506-b334e2"]
Sep 19 01:37:49.652956 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:49.652935    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7506-b334e2"
Sep 19 01:37:49.655371 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:49.655350    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-frkf9\""
Sep 19 01:37:49.661897 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:49.661878    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7506-b334e2"]
Sep 19 01:37:49.809784 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:49.809755    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j9df\" (UniqueName: \"kubernetes.io/projected/cac50260-ce71-46b8-b973-91bc32267314-kube-api-access-7j9df\") pod \"test-curl-7506-b334e2\" (UID: \"cac50260-ce71-46b8-b973-91bc32267314\") " pod="opendatahub/test-curl-7506-b334e2"
Sep 19 01:37:49.911276 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:49.911199    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j9df\" (UniqueName: \"kubernetes.io/projected/cac50260-ce71-46b8-b973-91bc32267314-kube-api-access-7j9df\") pod \"test-curl-7506-b334e2\" (UID: \"cac50260-ce71-46b8-b973-91bc32267314\") " pod="opendatahub/test-curl-7506-b334e2"
Sep 19 01:37:49.920558 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:49.920528    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j9df\" (UniqueName: \"kubernetes.io/projected/cac50260-ce71-46b8-b973-91bc32267314-kube-api-access-7j9df\") pod \"test-curl-7506-b334e2\" (UID: \"cac50260-ce71-46b8-b973-91bc32267314\") " pod="opendatahub/test-curl-7506-b334e2"
Sep 19 01:37:49.963227 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:49.963201    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7506-b334e2"
Sep 19 01:37:50.239093 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:37:50.239052    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcac50260_ce71_46b8_b973_91bc32267314.slice/crio-65764e761e332f884680109fce3536a0fe3194636f331044eae28b8c6604180d WatchSource:0}: Error finding container 65764e761e332f884680109fce3536a0fe3194636f331044eae28b8c6604180d: Status 404 returned error can't find the container with id 65764e761e332f884680109fce3536a0fe3194636f331044eae28b8c6604180d
Sep 19 01:37:50.239806 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:50.239777    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7506-b334e2"]
Sep 19 01:37:50.794321 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:50.794287    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7506-b334e2" event={"ID":"cac50260-ce71-46b8-b973-91bc32267314","Type":"ContainerStarted","Data":"4061dcf02fc03667e3c8e61166098e6297db2b8c686c269c3efea966c3a72a29"}
Sep 19 01:37:50.794321 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:50.794326    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7506-b334e2" event={"ID":"cac50260-ce71-46b8-b973-91bc32267314","Type":"ContainerStarted","Data":"65764e761e332f884680109fce3536a0fe3194636f331044eae28b8c6604180d"}
Sep 19 01:37:50.815504 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:50.815456    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7506-b334e2" podStartSLOduration=1.815441056 podStartE2EDuration="1.815441056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:37:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:37:50.811833844 +0000 UTC m=+1525.718695893" watchObservedRunningTime="2026-09-19 01:37:50.815441056 +0000 UTC m=+1525.722303158"
Sep 19 01:37:51.459997 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:51.459963    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7506-b334e2"]
Sep 19 01:37:51.466563 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:51.466533    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7506-b334e2"]
Sep 19 01:37:51.803715 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:37:51.803685    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7506-b334e2" container="test-curl-7506-b334e2"
Sep 19 01:37:52.810233 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:52.810191    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7506-b334e2" podUID="cac50260-ce71-46b8-b973-91bc32267314" containerName="test-curl-7506-b334e2" containerID="cri-o://4061dcf02fc03667e3c8e61166098e6297db2b8c686c269c3efea966c3a72a29" gracePeriod=2
Sep 19 01:37:52.813205 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:37:52.813172    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7506-b334e2\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="cac50260-ce71-46b8-b973-91bc32267314" pod="opendatahub/test-curl-7506-b334e2"
Sep 19 01:37:52.889772 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:52.889741    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"]
Sep 19 01:37:52.909788 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:37:52.909756    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7506-b334e2\" is forbidden: User \"system:node:ip-10-0-131-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-76.ec2.internal' and this object" podUID="cac50260-ce71-46b8-b973-91bc32267314" pod="opendatahub/test-curl-7506-b334e2"
Sep 19 01:37:53.899957 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:53.899926    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"]
Sep 19 01:37:54.973272 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:54.973249    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7506-b334e2"
Sep 19 01:37:55.065153 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:55.065129    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7j9df\" (UniqueName: \"kubernetes.io/projected/cac50260-ce71-46b8-b973-91bc32267314-kube-api-access-7j9df\") pod \"cac50260-ce71-46b8-b973-91bc32267314\" (UID: \"cac50260-ce71-46b8-b973-91bc32267314\") "
Sep 19 01:37:55.067063 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:55.067036    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cac50260-ce71-46b8-b973-91bc32267314-kube-api-access-7j9df" (OuterVolumeSpecName: "kube-api-access-7j9df") pod "cac50260-ce71-46b8-b973-91bc32267314" (UID: "cac50260-ce71-46b8-b973-91bc32267314"). InnerVolumeSpecName "kube-api-access-7j9df". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:37:55.166069 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:55.166008    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7j9df\" (UniqueName: \"kubernetes.io/projected/cac50260-ce71-46b8-b973-91bc32267314-kube-api-access-7j9df\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:37:55.834003 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:55.833969    2580 generic.go:356] "Generic (PLEG): container finished" podID="cac50260-ce71-46b8-b973-91bc32267314" containerID="4061dcf02fc03667e3c8e61166098e6297db2b8c686c269c3efea966c3a72a29" exitCode=137
Sep 19 01:37:55.834188 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:55.834021    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7506-b334e2"
Sep 19 01:37:55.834188 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:55.834075    2580 scope.go:117] "RemoveContainer" containerID="4061dcf02fc03667e3c8e61166098e6297db2b8c686c269c3efea966c3a72a29"
Sep 19 01:37:55.843958 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:55.843939    2580 scope.go:117] "RemoveContainer" containerID="4061dcf02fc03667e3c8e61166098e6297db2b8c686c269c3efea966c3a72a29"
Sep 19 01:37:55.844231 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:37:55.844213    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4061dcf02fc03667e3c8e61166098e6297db2b8c686c269c3efea966c3a72a29\": container with ID starting with 4061dcf02fc03667e3c8e61166098e6297db2b8c686c269c3efea966c3a72a29 not found: ID does not exist" containerID="4061dcf02fc03667e3c8e61166098e6297db2b8c686c269c3efea966c3a72a29"
Sep 19 01:37:55.844312 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:55.844238    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4061dcf02fc03667e3c8e61166098e6297db2b8c686c269c3efea966c3a72a29"} err="failed to get container status \"4061dcf02fc03667e3c8e61166098e6297db2b8c686c269c3efea966c3a72a29\": rpc error: code = NotFound desc = could not find container \"4061dcf02fc03667e3c8e61166098e6297db2b8c686c269c3efea966c3a72a29\": container with ID starting with 4061dcf02fc03667e3c8e61166098e6297db2b8c686c269c3efea966c3a72a29 not found: ID does not exist"
Sep 19 01:37:57.065769 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:57.065738    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cac50260-ce71-46b8-b973-91bc32267314" path="/var/lib/kubelet/pods/cac50260-ce71-46b8-b973-91bc32267314/volumes"
Sep 19 01:37:58.190813 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:58.190781    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw"]
Sep 19 01:37:58.191934 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:58.191858    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw" podUID="7be56cd0-b212-4565-b84f-4d994a978624" containerName="maas-api" containerID="cri-o://481cc039210fe9350e0946677f2fd53d17943ad4dd9f39dcb983db461a3a353c" gracePeriod=30
Sep 19 01:37:58.441927 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:58.441869    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw"
Sep 19 01:37:58.597274 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:58.597242    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7be56cd0-b212-4565-b84f-4d994a978624-maas-api-tls\") pod \"7be56cd0-b212-4565-b84f-4d994a978624\" (UID: \"7be56cd0-b212-4565-b84f-4d994a978624\") "
Sep 19 01:37:58.597440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:58.597286    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7be56cd0-b212-4565-b84f-4d994a978624-openshift-service-ca\") pod \"7be56cd0-b212-4565-b84f-4d994a978624\" (UID: \"7be56cd0-b212-4565-b84f-4d994a978624\") "
Sep 19 01:37:58.597440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:58.597345    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfznt\" (UniqueName: \"kubernetes.io/projected/7be56cd0-b212-4565-b84f-4d994a978624-kube-api-access-nfznt\") pod \"7be56cd0-b212-4565-b84f-4d994a978624\" (UID: \"7be56cd0-b212-4565-b84f-4d994a978624\") "
Sep 19 01:37:58.597440 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:58.597395    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7be56cd0-b212-4565-b84f-4d994a978624-metrics-certs\") pod \"7be56cd0-b212-4565-b84f-4d994a978624\" (UID: \"7be56cd0-b212-4565-b84f-4d994a978624\") "
Sep 19 01:37:58.597752 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:58.597728    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7be56cd0-b212-4565-b84f-4d994a978624-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7be56cd0-b212-4565-b84f-4d994a978624" (UID: "7be56cd0-b212-4565-b84f-4d994a978624"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:37:58.599477 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:58.599454    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7be56cd0-b212-4565-b84f-4d994a978624-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7be56cd0-b212-4565-b84f-4d994a978624" (UID: "7be56cd0-b212-4565-b84f-4d994a978624"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:37:58.599572 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:58.599551    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7be56cd0-b212-4565-b84f-4d994a978624-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7be56cd0-b212-4565-b84f-4d994a978624" (UID: "7be56cd0-b212-4565-b84f-4d994a978624"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:37:58.599621 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:58.599570    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7be56cd0-b212-4565-b84f-4d994a978624-kube-api-access-nfznt" (OuterVolumeSpecName: "kube-api-access-nfznt") pod "7be56cd0-b212-4565-b84f-4d994a978624" (UID: "7be56cd0-b212-4565-b84f-4d994a978624"). InnerVolumeSpecName "kube-api-access-nfznt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:37:58.698525 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:58.698428    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7be56cd0-b212-4565-b84f-4d994a978624-metrics-certs\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:37:58.698525 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:58.698453    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7be56cd0-b212-4565-b84f-4d994a978624-maas-api-tls\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:37:58.698525 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:58.698466    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7be56cd0-b212-4565-b84f-4d994a978624-openshift-service-ca\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:37:58.698525 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:58.698475    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfznt\" (UniqueName: \"kubernetes.io/projected/7be56cd0-b212-4565-b84f-4d994a978624-kube-api-access-nfznt\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:37:58.855590 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:58.855555    2580 generic.go:356] "Generic (PLEG): container finished" podID="7be56cd0-b212-4565-b84f-4d994a978624" containerID="481cc039210fe9350e0946677f2fd53d17943ad4dd9f39dcb983db461a3a353c" exitCode=0
Sep 19 01:37:58.855774 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:58.855597    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw"
Sep 19 01:37:58.855774 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:58.855681    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw" event={"ID":"7be56cd0-b212-4565-b84f-4d994a978624","Type":"ContainerDied","Data":"481cc039210fe9350e0946677f2fd53d17943ad4dd9f39dcb983db461a3a353c"}
Sep 19 01:37:58.855774 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:58.855736    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw" event={"ID":"7be56cd0-b212-4565-b84f-4d994a978624","Type":"ContainerDied","Data":"d0f5af8db7079a2951db9587eadff9bf3669a4d69a570fb0e7f4b502bfa04685"}
Sep 19 01:37:58.855774 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:58.855766    2580 scope.go:117] "RemoveContainer" containerID="481cc039210fe9350e0946677f2fd53d17943ad4dd9f39dcb983db461a3a353c"
Sep 19 01:37:58.865622 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:58.865605    2580 scope.go:117] "RemoveContainer" containerID="481cc039210fe9350e0946677f2fd53d17943ad4dd9f39dcb983db461a3a353c"
Sep 19 01:37:58.865893 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:37:58.865875    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"481cc039210fe9350e0946677f2fd53d17943ad4dd9f39dcb983db461a3a353c\": container with ID starting with 481cc039210fe9350e0946677f2fd53d17943ad4dd9f39dcb983db461a3a353c not found: ID does not exist" containerID="481cc039210fe9350e0946677f2fd53d17943ad4dd9f39dcb983db461a3a353c"
Sep 19 01:37:58.865963 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:58.865908    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"481cc039210fe9350e0946677f2fd53d17943ad4dd9f39dcb983db461a3a353c"} err="failed to get container status \"481cc039210fe9350e0946677f2fd53d17943ad4dd9f39dcb983db461a3a353c\": rpc error: code = NotFound desc = could not find container \"481cc039210fe9350e0946677f2fd53d17943ad4dd9f39dcb983db461a3a353c\": container with ID starting with 481cc039210fe9350e0946677f2fd53d17943ad4dd9f39dcb983db461a3a353c not found: ID does not exist"
Sep 19 01:37:58.887593 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:58.887567    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw"]
Sep 19 01:37:58.894079 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:58.894059    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7b600-5c4fd6669c-bk5dw"]
Sep 19 01:37:59.064691 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:37:59.064664    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7be56cd0-b212-4565-b84f-4d994a978624" path="/var/lib/kubelet/pods/7be56cd0-b212-4565-b84f-4d994a978624/volumes"
Sep 19 01:38:07.910428 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:07.910385    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh" podUID="33f30bb4-e89e-4c99-9801-787321b0a13d" containerName="main" containerID="cri-o://c796809a33f774030333ee911139e1fb44056feada10aecb77cada87575610b7" gracePeriod=45
Sep 19 01:38:08.171713 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.171657    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:38:08.183539 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.183510    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33f30bb4-e89e-4c99-9801-787321b0a13d-tmp-dir\") pod \"33f30bb4-e89e-4c99-9801-787321b0a13d\" (UID: \"33f30bb4-e89e-4c99-9801-787321b0a13d\") "
Sep 19 01:38:08.183694 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.183542    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33f30bb4-e89e-4c99-9801-787321b0a13d-tls-certs\") pod \"33f30bb4-e89e-4c99-9801-787321b0a13d\" (UID: \"33f30bb4-e89e-4c99-9801-787321b0a13d\") "
Sep 19 01:38:08.183694 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.183600    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33f30bb4-e89e-4c99-9801-787321b0a13d-home\") pod \"33f30bb4-e89e-4c99-9801-787321b0a13d\" (UID: \"33f30bb4-e89e-4c99-9801-787321b0a13d\") "
Sep 19 01:38:08.183694 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.183653    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68szp\" (UniqueName: \"kubernetes.io/projected/33f30bb4-e89e-4c99-9801-787321b0a13d-kube-api-access-68szp\") pod \"33f30bb4-e89e-4c99-9801-787321b0a13d\" (UID: \"33f30bb4-e89e-4c99-9801-787321b0a13d\") "
Sep 19 01:38:08.183885 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.183723    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33f30bb4-e89e-4c99-9801-787321b0a13d-model-cache\") pod \"33f30bb4-e89e-4c99-9801-787321b0a13d\" (UID: \"33f30bb4-e89e-4c99-9801-787321b0a13d\") "
Sep 19 01:38:08.183885 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.183738    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33f30bb4-e89e-4c99-9801-787321b0a13d-dshm\") pod \"33f30bb4-e89e-4c99-9801-787321b0a13d\" (UID: \"33f30bb4-e89e-4c99-9801-787321b0a13d\") "
Sep 19 01:38:08.183885 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.183809    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33f30bb4-e89e-4c99-9801-787321b0a13d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "33f30bb4-e89e-4c99-9801-787321b0a13d" (UID: "33f30bb4-e89e-4c99-9801-787321b0a13d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:38:08.184132 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.184073    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33f30bb4-e89e-4c99-9801-787321b0a13d-model-cache" (OuterVolumeSpecName: "model-cache") pod "33f30bb4-e89e-4c99-9801-787321b0a13d" (UID: "33f30bb4-e89e-4c99-9801-787321b0a13d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:38:08.184132 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.184086    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33f30bb4-e89e-4c99-9801-787321b0a13d-tmp-dir\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:38:08.184132 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.184105    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33f30bb4-e89e-4c99-9801-787321b0a13d-home" (OuterVolumeSpecName: "home") pod "33f30bb4-e89e-4c99-9801-787321b0a13d" (UID: "33f30bb4-e89e-4c99-9801-787321b0a13d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:38:08.186118 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.186095    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33f30bb4-e89e-4c99-9801-787321b0a13d-dshm" (OuterVolumeSpecName: "dshm") pod "33f30bb4-e89e-4c99-9801-787321b0a13d" (UID: "33f30bb4-e89e-4c99-9801-787321b0a13d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:38:08.186232 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.186118    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33f30bb4-e89e-4c99-9801-787321b0a13d-kube-api-access-68szp" (OuterVolumeSpecName: "kube-api-access-68szp") pod "33f30bb4-e89e-4c99-9801-787321b0a13d" (UID: "33f30bb4-e89e-4c99-9801-787321b0a13d"). InnerVolumeSpecName "kube-api-access-68szp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:38:08.186232 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.186203    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33f30bb4-e89e-4c99-9801-787321b0a13d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "33f30bb4-e89e-4c99-9801-787321b0a13d" (UID: "33f30bb4-e89e-4c99-9801-787321b0a13d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:38:08.285411 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.285373    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33f30bb4-e89e-4c99-9801-787321b0a13d-model-cache\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:38:08.285411 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.285408    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33f30bb4-e89e-4c99-9801-787321b0a13d-dshm\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:38:08.285662 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.285422    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33f30bb4-e89e-4c99-9801-787321b0a13d-tls-certs\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:38:08.285662 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.285434    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33f30bb4-e89e-4c99-9801-787321b0a13d-home\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:38:08.285662 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.285447    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68szp\" (UniqueName: \"kubernetes.io/projected/33f30bb4-e89e-4c99-9801-787321b0a13d-kube-api-access-68szp\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:38:08.919890 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.919853    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps" podUID="2c7bac26-42b5-4b3d-9a03-8205d2691f96" containerName="main" containerID="cri-o://035a7edb69d2120f03e368fe6d571586f6cc56e3080208ac40acf1904d0909ae" gracePeriod=45
Sep 19 01:38:08.930034 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.930002    2580 generic.go:356] "Generic (PLEG): container finished" podID="33f30bb4-e89e-4c99-9801-787321b0a13d" containerID="c796809a33f774030333ee911139e1fb44056feada10aecb77cada87575610b7" exitCode=0
Sep 19 01:38:08.930158 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.930043    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"
Sep 19 01:38:08.930158 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.930077    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh" event={"ID":"33f30bb4-e89e-4c99-9801-787321b0a13d","Type":"ContainerDied","Data":"c796809a33f774030333ee911139e1fb44056feada10aecb77cada87575610b7"}
Sep 19 01:38:08.930158 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.930117    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh" event={"ID":"33f30bb4-e89e-4c99-9801-787321b0a13d","Type":"ContainerDied","Data":"faa465477c7602de863860b150f7d184bc6c89a23a11d10911cdaf7e40215928"}
Sep 19 01:38:08.930158 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.930134    2580 scope.go:117] "RemoveContainer" containerID="c796809a33f774030333ee911139e1fb44056feada10aecb77cada87575610b7"
Sep 19 01:38:08.998889 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:08.998857    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"]
Sep 19 01:38:09.002858 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.002834    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-751f6a/auth-test-model-751f6a-kserve-888dfd979-fb5jh"]
Sep 19 01:38:09.034358 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.034337    2580 scope.go:117] "RemoveContainer" containerID="c796809a33f774030333ee911139e1fb44056feada10aecb77cada87575610b7"
Sep 19 01:38:09.034793 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:38:09.034770    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c796809a33f774030333ee911139e1fb44056feada10aecb77cada87575610b7\": container with ID starting with c796809a33f774030333ee911139e1fb44056feada10aecb77cada87575610b7 not found: ID does not exist" containerID="c796809a33f774030333ee911139e1fb44056feada10aecb77cada87575610b7"
Sep 19 01:38:09.034901 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.034805    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c796809a33f774030333ee911139e1fb44056feada10aecb77cada87575610b7"} err="failed to get container status \"c796809a33f774030333ee911139e1fb44056feada10aecb77cada87575610b7\": rpc error: code = NotFound desc = could not find container \"c796809a33f774030333ee911139e1fb44056feada10aecb77cada87575610b7\": container with ID starting with c796809a33f774030333ee911139e1fb44056feada10aecb77cada87575610b7 not found: ID does not exist"
Sep 19 01:38:09.065310 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.065276    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33f30bb4-e89e-4c99-9801-787321b0a13d" path="/var/lib/kubelet/pods/33f30bb4-e89e-4c99-9801-787321b0a13d/volumes"
Sep 19 01:38:09.192589 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.192565    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:38:09.295793 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.295763    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c7bac26-42b5-4b3d-9a03-8205d2691f96-model-cache\") pod \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\" (UID: \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\") "
Sep 19 01:38:09.295968 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.295805    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c7bac26-42b5-4b3d-9a03-8205d2691f96-tls-certs\") pod \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\" (UID: \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\") "
Sep 19 01:38:09.295968 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.295828    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c7bac26-42b5-4b3d-9a03-8205d2691f96-tmp-dir\") pod \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\" (UID: \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\") "
Sep 19 01:38:09.295968 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.295850    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvlhq\" (UniqueName: \"kubernetes.io/projected/2c7bac26-42b5-4b3d-9a03-8205d2691f96-kube-api-access-pvlhq\") pod \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\" (UID: \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\") "
Sep 19 01:38:09.295968 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.295907    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c7bac26-42b5-4b3d-9a03-8205d2691f96-home\") pod \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\" (UID: \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\") "
Sep 19 01:38:09.295968 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.295949    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c7bac26-42b5-4b3d-9a03-8205d2691f96-dshm\") pod \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\" (UID: \"2c7bac26-42b5-4b3d-9a03-8205d2691f96\") "
Sep 19 01:38:09.296246 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.296029    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c7bac26-42b5-4b3d-9a03-8205d2691f96-model-cache" (OuterVolumeSpecName: "model-cache") pod "2c7bac26-42b5-4b3d-9a03-8205d2691f96" (UID: "2c7bac26-42b5-4b3d-9a03-8205d2691f96"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:38:09.296246 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.296046    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c7bac26-42b5-4b3d-9a03-8205d2691f96-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2c7bac26-42b5-4b3d-9a03-8205d2691f96" (UID: "2c7bac26-42b5-4b3d-9a03-8205d2691f96"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:38:09.296246 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.296124    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c7bac26-42b5-4b3d-9a03-8205d2691f96-home" (OuterVolumeSpecName: "home") pod "2c7bac26-42b5-4b3d-9a03-8205d2691f96" (UID: "2c7bac26-42b5-4b3d-9a03-8205d2691f96"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:38:09.296350 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.296275    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c7bac26-42b5-4b3d-9a03-8205d2691f96-home\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:38:09.296350 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.296289    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c7bac26-42b5-4b3d-9a03-8205d2691f96-model-cache\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:38:09.296350 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.296301    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c7bac26-42b5-4b3d-9a03-8205d2691f96-tmp-dir\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:38:09.298097 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.298072    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c7bac26-42b5-4b3d-9a03-8205d2691f96-dshm" (OuterVolumeSpecName: "dshm") pod "2c7bac26-42b5-4b3d-9a03-8205d2691f96" (UID: "2c7bac26-42b5-4b3d-9a03-8205d2691f96"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:38:09.298203 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.298113    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c7bac26-42b5-4b3d-9a03-8205d2691f96-kube-api-access-pvlhq" (OuterVolumeSpecName: "kube-api-access-pvlhq") pod "2c7bac26-42b5-4b3d-9a03-8205d2691f96" (UID: "2c7bac26-42b5-4b3d-9a03-8205d2691f96"). InnerVolumeSpecName "kube-api-access-pvlhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:38:09.298203 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.298122    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c7bac26-42b5-4b3d-9a03-8205d2691f96-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2c7bac26-42b5-4b3d-9a03-8205d2691f96" (UID: "2c7bac26-42b5-4b3d-9a03-8205d2691f96"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:38:09.397045 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.397010    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c7bac26-42b5-4b3d-9a03-8205d2691f96-tls-certs\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:38:09.397045 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.397038    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvlhq\" (UniqueName: \"kubernetes.io/projected/2c7bac26-42b5-4b3d-9a03-8205d2691f96-kube-api-access-pvlhq\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:38:09.397045 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.397047    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c7bac26-42b5-4b3d-9a03-8205d2691f96-dshm\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:38:09.940204 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.940165    2580 generic.go:356] "Generic (PLEG): container finished" podID="2c7bac26-42b5-4b3d-9a03-8205d2691f96" containerID="035a7edb69d2120f03e368fe6d571586f6cc56e3080208ac40acf1904d0909ae" exitCode=0
Sep 19 01:38:09.940675 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.940206    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"
Sep 19 01:38:09.940675 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.940238    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps" event={"ID":"2c7bac26-42b5-4b3d-9a03-8205d2691f96","Type":"ContainerDied","Data":"035a7edb69d2120f03e368fe6d571586f6cc56e3080208ac40acf1904d0909ae"}
Sep 19 01:38:09.940675 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.940274    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps" event={"ID":"2c7bac26-42b5-4b3d-9a03-8205d2691f96","Type":"ContainerDied","Data":"cb6afaa82cae1e577a90e45d6bdbb0ec1c9674077992114b066caafea3b79aa6"}
Sep 19 01:38:09.940675 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.940296    2580 scope.go:117] "RemoveContainer" containerID="035a7edb69d2120f03e368fe6d571586f6cc56e3080208ac40acf1904d0909ae"
Sep 19 01:38:09.951085 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.951063    2580 scope.go:117] "RemoveContainer" containerID="035a7edb69d2120f03e368fe6d571586f6cc56e3080208ac40acf1904d0909ae"
Sep 19 01:38:09.951350 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:38:09.951331    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"035a7edb69d2120f03e368fe6d571586f6cc56e3080208ac40acf1904d0909ae\": container with ID starting with 035a7edb69d2120f03e368fe6d571586f6cc56e3080208ac40acf1904d0909ae not found: ID does not exist" containerID="035a7edb69d2120f03e368fe6d571586f6cc56e3080208ac40acf1904d0909ae"
Sep 19 01:38:09.951398 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.951358    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"035a7edb69d2120f03e368fe6d571586f6cc56e3080208ac40acf1904d0909ae"} err="failed to get container status \"035a7edb69d2120f03e368fe6d571586f6cc56e3080208ac40acf1904d0909ae\": rpc error: code = NotFound desc = could not find container \"035a7edb69d2120f03e368fe6d571586f6cc56e3080208ac40acf1904d0909ae\": container with ID starting with 035a7edb69d2120f03e368fe6d571586f6cc56e3080208ac40acf1904d0909ae not found: ID does not exist"
Sep 19 01:38:09.965570 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.965546    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"]
Sep 19 01:38:09.969679 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:09.969660    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-bebef4/auth-test-model-bebef4-kserve-f7474ccd7-qgmps"]
Sep 19 01:38:11.064398 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:11.064365    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c7bac26-42b5-4b3d-9a03-8205d2691f96" path="/var/lib/kubelet/pods/2c7bac26-42b5-4b3d-9a03-8205d2691f96/volumes"
Sep 19 01:38:21.579114 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:21.579081    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7"]
Sep 19 01:38:21.579638 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:21.579302    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7" podUID="c834b565-3fe9-4da0-8f6f-608c0f015c5a" containerName="maas-api" containerID="cri-o://ee2d0628faa699ca27d7523d737704b905109a90ca62d607fc2de3a2b03b89b0" gracePeriod=30
Sep 19 01:38:21.843380 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:21.843314    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7"
Sep 19 01:38:21.919435 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:21.919398    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c834b565-3fe9-4da0-8f6f-608c0f015c5a-maas-api-tls\") pod \"c834b565-3fe9-4da0-8f6f-608c0f015c5a\" (UID: \"c834b565-3fe9-4da0-8f6f-608c0f015c5a\") "
Sep 19 01:38:21.919619 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:21.919478    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c834b565-3fe9-4da0-8f6f-608c0f015c5a-openshift-service-ca\") pod \"c834b565-3fe9-4da0-8f6f-608c0f015c5a\" (UID: \"c834b565-3fe9-4da0-8f6f-608c0f015c5a\") "
Sep 19 01:38:21.919619 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:21.919533    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c834b565-3fe9-4da0-8f6f-608c0f015c5a-metrics-certs\") pod \"c834b565-3fe9-4da0-8f6f-608c0f015c5a\" (UID: \"c834b565-3fe9-4da0-8f6f-608c0f015c5a\") "
Sep 19 01:38:21.919619 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:21.919562    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slfwz\" (UniqueName: \"kubernetes.io/projected/c834b565-3fe9-4da0-8f6f-608c0f015c5a-kube-api-access-slfwz\") pod \"c834b565-3fe9-4da0-8f6f-608c0f015c5a\" (UID: \"c834b565-3fe9-4da0-8f6f-608c0f015c5a\") "
Sep 19 01:38:21.919833 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:21.919809    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c834b565-3fe9-4da0-8f6f-608c0f015c5a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c834b565-3fe9-4da0-8f6f-608c0f015c5a" (UID: "c834b565-3fe9-4da0-8f6f-608c0f015c5a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:38:21.919946 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:21.919926    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c834b565-3fe9-4da0-8f6f-608c0f015c5a-openshift-service-ca\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:38:21.921759 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:21.921727    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c834b565-3fe9-4da0-8f6f-608c0f015c5a-kube-api-access-slfwz" (OuterVolumeSpecName: "kube-api-access-slfwz") pod "c834b565-3fe9-4da0-8f6f-608c0f015c5a" (UID: "c834b565-3fe9-4da0-8f6f-608c0f015c5a"). InnerVolumeSpecName "kube-api-access-slfwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:38:21.921894 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:21.921734    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c834b565-3fe9-4da0-8f6f-608c0f015c5a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c834b565-3fe9-4da0-8f6f-608c0f015c5a" (UID: "c834b565-3fe9-4da0-8f6f-608c0f015c5a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:38:21.921894 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:21.921782    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c834b565-3fe9-4da0-8f6f-608c0f015c5a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c834b565-3fe9-4da0-8f6f-608c0f015c5a" (UID: "c834b565-3fe9-4da0-8f6f-608c0f015c5a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:38:22.017892 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:22.017857    2580 generic.go:356] "Generic (PLEG): container finished" podID="c834b565-3fe9-4da0-8f6f-608c0f015c5a" containerID="ee2d0628faa699ca27d7523d737704b905109a90ca62d607fc2de3a2b03b89b0" exitCode=0
Sep 19 01:38:22.018054 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:22.017908    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7"
Sep 19 01:38:22.018054 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:22.017939    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7" event={"ID":"c834b565-3fe9-4da0-8f6f-608c0f015c5a","Type":"ContainerDied","Data":"ee2d0628faa699ca27d7523d737704b905109a90ca62d607fc2de3a2b03b89b0"}
Sep 19 01:38:22.018054 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:22.017989    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7" event={"ID":"c834b565-3fe9-4da0-8f6f-608c0f015c5a","Type":"ContainerDied","Data":"d38d8920fe703f0597fd3c1a34c465937f4b72417fd29a7362676132963beb8d"}
Sep 19 01:38:22.018054 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:22.018013    2580 scope.go:117] "RemoveContainer" containerID="ee2d0628faa699ca27d7523d737704b905109a90ca62d607fc2de3a2b03b89b0"
Sep 19 01:38:22.020428 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:22.020396    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c834b565-3fe9-4da0-8f6f-608c0f015c5a-metrics-certs\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:38:22.020428 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:22.020417    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slfwz\" (UniqueName: \"kubernetes.io/projected/c834b565-3fe9-4da0-8f6f-608c0f015c5a-kube-api-access-slfwz\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:38:22.020428 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:22.020426    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c834b565-3fe9-4da0-8f6f-608c0f015c5a-maas-api-tls\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:38:22.029173 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:22.029065    2580 scope.go:117] "RemoveContainer" containerID="ee2d0628faa699ca27d7523d737704b905109a90ca62d607fc2de3a2b03b89b0"
Sep 19 01:38:22.029424 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:38:22.029401    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee2d0628faa699ca27d7523d737704b905109a90ca62d607fc2de3a2b03b89b0\": container with ID starting with ee2d0628faa699ca27d7523d737704b905109a90ca62d607fc2de3a2b03b89b0 not found: ID does not exist" containerID="ee2d0628faa699ca27d7523d737704b905109a90ca62d607fc2de3a2b03b89b0"
Sep 19 01:38:22.029612 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:22.029582    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee2d0628faa699ca27d7523d737704b905109a90ca62d607fc2de3a2b03b89b0"} err="failed to get container status \"ee2d0628faa699ca27d7523d737704b905109a90ca62d607fc2de3a2b03b89b0\": rpc error: code = NotFound desc = could not find container \"ee2d0628faa699ca27d7523d737704b905109a90ca62d607fc2de3a2b03b89b0\": container with ID starting with ee2d0628faa699ca27d7523d737704b905109a90ca62d607fc2de3a2b03b89b0 not found: ID does not exist"
Sep 19 01:38:22.049757 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:22.049731    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7"]
Sep 19 01:38:22.052383 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:22.052364    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-29d76c-77f7b8c5bf-msrc7"]
Sep 19 01:38:23.063807 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:23.063777    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c834b565-3fe9-4da0-8f6f-608c0f015c5a" path="/var/lib/kubelet/pods/c834b565-3fe9-4da0-8f6f-608c0f015c5a/volumes"
Sep 19 01:38:39.366726 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:39.366687    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx"]
Sep 19 01:38:39.367528 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:39.367452    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx" podUID="1c0d4ade-2a79-4833-8c42-306dd75927c6" containerName="maas-api" containerID="cri-o://15e2c0e43e6f08822f86c48947444516ee1f7940097b929520c1e6ad2d46e4a9" gracePeriod=30
Sep 19 01:38:39.446275 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:39.446244    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj"]
Sep 19 01:38:39.446506 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:39.446451    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj" podUID="ca421c49-d4f8-477c-8974-70ffc05b9468" containerName="payload-pre-processing" containerID="cri-o://bd62a794981dee7fc6876b0638a87ff063efe73bd026f7505bf02da13e9edc74" gracePeriod=30
Sep 19 01:38:39.710191 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:39.710165    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx"
Sep 19 01:38:39.750064 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:39.750038    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj"
Sep 19 01:38:39.898654 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:39.898572    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c0d4ade-2a79-4833-8c42-306dd75927c6-metrics-certs\") pod \"1c0d4ade-2a79-4833-8c42-306dd75927c6\" (UID: \"1c0d4ade-2a79-4833-8c42-306dd75927c6\") "
Sep 19 01:38:39.898654 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:39.898630    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c0d4ade-2a79-4833-8c42-306dd75927c6-openshift-service-ca\") pod \"1c0d4ade-2a79-4833-8c42-306dd75927c6\" (UID: \"1c0d4ade-2a79-4833-8c42-306dd75927c6\") "
Sep 19 01:38:39.898845 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:39.898686    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvrj4\" (UniqueName: \"kubernetes.io/projected/ca421c49-d4f8-477c-8974-70ffc05b9468-kube-api-access-cvrj4\") pod \"ca421c49-d4f8-477c-8974-70ffc05b9468\" (UID: \"ca421c49-d4f8-477c-8974-70ffc05b9468\") "
Sep 19 01:38:39.898845 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:39.898709    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca421c49-d4f8-477c-8974-70ffc05b9468-plugins-config-volume\") pod \"ca421c49-d4f8-477c-8974-70ffc05b9468\" (UID: \"ca421c49-d4f8-477c-8974-70ffc05b9468\") "
Sep 19 01:38:39.898946 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:39.898842    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c0d4ade-2a79-4833-8c42-306dd75927c6-maas-api-tls\") pod \"1c0d4ade-2a79-4833-8c42-306dd75927c6\" (UID: \"1c0d4ade-2a79-4833-8c42-306dd75927c6\") "
Sep 19 01:38:39.899004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:39.898942    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vsp2s\" (UniqueName: \"kubernetes.io/projected/1c0d4ade-2a79-4833-8c42-306dd75927c6-kube-api-access-vsp2s\") pod \"1c0d4ade-2a79-4833-8c42-306dd75927c6\" (UID: \"1c0d4ade-2a79-4833-8c42-306dd75927c6\") "
Sep 19 01:38:39.899062 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:39.899034    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca421c49-d4f8-477c-8974-70ffc05b9468-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ca421c49-d4f8-477c-8974-70ffc05b9468" (UID: "ca421c49-d4f8-477c-8974-70ffc05b9468"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:38:39.899122 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:39.899054    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c0d4ade-2a79-4833-8c42-306dd75927c6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1c0d4ade-2a79-4833-8c42-306dd75927c6" (UID: "1c0d4ade-2a79-4833-8c42-306dd75927c6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:38:39.899330 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:39.899307    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca421c49-d4f8-477c-8974-70ffc05b9468-plugins-config-volume\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:38:39.899330 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:39.899332    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c0d4ade-2a79-4833-8c42-306dd75927c6-openshift-service-ca\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:38:39.900793 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:39.900770    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c0d4ade-2a79-4833-8c42-306dd75927c6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1c0d4ade-2a79-4833-8c42-306dd75927c6" (UID: "1c0d4ade-2a79-4833-8c42-306dd75927c6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:38:39.900876 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:39.900811    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca421c49-d4f8-477c-8974-70ffc05b9468-kube-api-access-cvrj4" (OuterVolumeSpecName: "kube-api-access-cvrj4") pod "ca421c49-d4f8-477c-8974-70ffc05b9468" (UID: "ca421c49-d4f8-477c-8974-70ffc05b9468"). InnerVolumeSpecName "kube-api-access-cvrj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:38:39.900924 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:39.900888    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c0d4ade-2a79-4833-8c42-306dd75927c6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1c0d4ade-2a79-4833-8c42-306dd75927c6" (UID: "1c0d4ade-2a79-4833-8c42-306dd75927c6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:38:39.900994 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:39.900976    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c0d4ade-2a79-4833-8c42-306dd75927c6-kube-api-access-vsp2s" (OuterVolumeSpecName: "kube-api-access-vsp2s") pod "1c0d4ade-2a79-4833-8c42-306dd75927c6" (UID: "1c0d4ade-2a79-4833-8c42-306dd75927c6"). InnerVolumeSpecName "kube-api-access-vsp2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:38:40.000708 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:40.000675    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c0d4ade-2a79-4833-8c42-306dd75927c6-metrics-certs\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:38:40.000708 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:40.000704    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvrj4\" (UniqueName: \"kubernetes.io/projected/ca421c49-d4f8-477c-8974-70ffc05b9468-kube-api-access-cvrj4\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:38:40.000708 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:40.000714    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c0d4ade-2a79-4833-8c42-306dd75927c6-maas-api-tls\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:38:40.000940 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:40.000723    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vsp2s\" (UniqueName: \"kubernetes.io/projected/1c0d4ade-2a79-4833-8c42-306dd75927c6-kube-api-access-vsp2s\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:38:40.132850 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:40.132817    2580 generic.go:356] "Generic (PLEG): container finished" podID="ca421c49-d4f8-477c-8974-70ffc05b9468" containerID="bd62a794981dee7fc6876b0638a87ff063efe73bd026f7505bf02da13e9edc74" exitCode=0
Sep 19 01:38:40.133036 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:40.132867    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj"
Sep 19 01:38:40.133036 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:40.132906    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj" event={"ID":"ca421c49-d4f8-477c-8974-70ffc05b9468","Type":"ContainerDied","Data":"bd62a794981dee7fc6876b0638a87ff063efe73bd026f7505bf02da13e9edc74"}
Sep 19 01:38:40.133036 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:40.132966    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj" event={"ID":"ca421c49-d4f8-477c-8974-70ffc05b9468","Type":"ContainerDied","Data":"e1126e93177e0777f829a0aded8c92112611faa37d1c6713c402ed008ec8c009"}
Sep 19 01:38:40.133036 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:40.132995    2580 scope.go:117] "RemoveContainer" containerID="bd62a794981dee7fc6876b0638a87ff063efe73bd026f7505bf02da13e9edc74"
Sep 19 01:38:40.134420 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:40.134398    2580 generic.go:356] "Generic (PLEG): container finished" podID="1c0d4ade-2a79-4833-8c42-306dd75927c6" containerID="15e2c0e43e6f08822f86c48947444516ee1f7940097b929520c1e6ad2d46e4a9" exitCode=0
Sep 19 01:38:40.134562 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:40.134446    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx" event={"ID":"1c0d4ade-2a79-4833-8c42-306dd75927c6","Type":"ContainerDied","Data":"15e2c0e43e6f08822f86c48947444516ee1f7940097b929520c1e6ad2d46e4a9"}
Sep 19 01:38:40.134562 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:40.134471    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx"
Sep 19 01:38:40.134562 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:40.134520    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx" event={"ID":"1c0d4ade-2a79-4833-8c42-306dd75927c6","Type":"ContainerDied","Data":"7703d23fe0d24ecb85e2c8069b52a53111b28033f80b819d96d2fcdb62510bbc"}
Sep 19 01:38:40.146269 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:40.146241    2580 scope.go:117] "RemoveContainer" containerID="bd62a794981dee7fc6876b0638a87ff063efe73bd026f7505bf02da13e9edc74"
Sep 19 01:38:40.146723 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:38:40.146695    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd62a794981dee7fc6876b0638a87ff063efe73bd026f7505bf02da13e9edc74\": container with ID starting with bd62a794981dee7fc6876b0638a87ff063efe73bd026f7505bf02da13e9edc74 not found: ID does not exist" containerID="bd62a794981dee7fc6876b0638a87ff063efe73bd026f7505bf02da13e9edc74"
Sep 19 01:38:40.146829 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:40.146734    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd62a794981dee7fc6876b0638a87ff063efe73bd026f7505bf02da13e9edc74"} err="failed to get container status \"bd62a794981dee7fc6876b0638a87ff063efe73bd026f7505bf02da13e9edc74\": rpc error: code = NotFound desc = could not find container \"bd62a794981dee7fc6876b0638a87ff063efe73bd026f7505bf02da13e9edc74\": container with ID starting with bd62a794981dee7fc6876b0638a87ff063efe73bd026f7505bf02da13e9edc74 not found: ID does not exist"
Sep 19 01:38:40.146829 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:40.146758    2580 scope.go:117] "RemoveContainer" containerID="15e2c0e43e6f08822f86c48947444516ee1f7940097b929520c1e6ad2d46e4a9"
Sep 19 01:38:40.157191 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:40.157176    2580 scope.go:117] "RemoveContainer" containerID="15e2c0e43e6f08822f86c48947444516ee1f7940097b929520c1e6ad2d46e4a9"
Sep 19 01:38:40.157609 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:38:40.157577    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15e2c0e43e6f08822f86c48947444516ee1f7940097b929520c1e6ad2d46e4a9\": container with ID starting with 15e2c0e43e6f08822f86c48947444516ee1f7940097b929520c1e6ad2d46e4a9 not found: ID does not exist" containerID="15e2c0e43e6f08822f86c48947444516ee1f7940097b929520c1e6ad2d46e4a9"
Sep 19 01:38:40.157712 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:40.157614    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15e2c0e43e6f08822f86c48947444516ee1f7940097b929520c1e6ad2d46e4a9"} err="failed to get container status \"15e2c0e43e6f08822f86c48947444516ee1f7940097b929520c1e6ad2d46e4a9\": rpc error: code = NotFound desc = could not find container \"15e2c0e43e6f08822f86c48947444516ee1f7940097b929520c1e6ad2d46e4a9\": container with ID starting with 15e2c0e43e6f08822f86c48947444516ee1f7940097b929520c1e6ad2d46e4a9 not found: ID does not exist"
Sep 19 01:38:40.165717 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:40.165689    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx"]
Sep 19 01:38:40.169233 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:40.169211    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-10325b-85b7cb5c97-2rrnx"]
Sep 19 01:38:40.178811 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:40.178789    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj"]
Sep 19 01:38:40.193296 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:40.193273    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-10325b-fcdcc97c4-fvdmj"]
Sep 19 01:38:41.063719 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:41.063680    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c0d4ade-2a79-4833-8c42-306dd75927c6" path="/var/lib/kubelet/pods/1c0d4ade-2a79-4833-8c42-306dd75927c6/volumes"
Sep 19 01:38:41.064125 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:38:41.064110    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca421c49-d4f8-477c-8974-70ffc05b9468" path="/var/lib/kubelet/pods/ca421c49-d4f8-477c-8974-70ffc05b9468/volumes"
Sep 19 01:39:06.718436 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.718402    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7506-e8c0be"]
Sep 19 01:39:06.718913 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.718899    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca421c49-d4f8-477c-8974-70ffc05b9468" containerName="payload-pre-processing"
Sep 19 01:39:06.718966 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.718915    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca421c49-d4f8-477c-8974-70ffc05b9468" containerName="payload-pre-processing"
Sep 19 01:39:06.718966 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.718924    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cac50260-ce71-46b8-b973-91bc32267314" containerName="test-curl-7506-b334e2"
Sep 19 01:39:06.718966 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.718930    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cac50260-ce71-46b8-b973-91bc32267314" containerName="test-curl-7506-b334e2"
Sep 19 01:39:06.718966 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.718940    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33f30bb4-e89e-4c99-9801-787321b0a13d" containerName="main"
Sep 19 01:39:06.718966 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.718945    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="33f30bb4-e89e-4c99-9801-787321b0a13d" containerName="main"
Sep 19 01:39:06.718966 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.718953    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c834b565-3fe9-4da0-8f6f-608c0f015c5a" containerName="maas-api"
Sep 19 01:39:06.718966 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.718958    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c834b565-3fe9-4da0-8f6f-608c0f015c5a" containerName="maas-api"
Sep 19 01:39:06.718966 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.718966    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c7bac26-42b5-4b3d-9a03-8205d2691f96" containerName="main"
Sep 19 01:39:06.718966 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.718971    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c7bac26-42b5-4b3d-9a03-8205d2691f96" containerName="main"
Sep 19 01:39:06.719242 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.718986    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7be56cd0-b212-4565-b84f-4d994a978624" containerName="maas-api"
Sep 19 01:39:06.719242 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.718992    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7be56cd0-b212-4565-b84f-4d994a978624" containerName="maas-api"
Sep 19 01:39:06.719242 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.718998    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c0d4ade-2a79-4833-8c42-306dd75927c6" containerName="maas-api"
Sep 19 01:39:06.719242 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.719003    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c0d4ade-2a79-4833-8c42-306dd75927c6" containerName="maas-api"
Sep 19 01:39:06.719242 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.719073    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7be56cd0-b212-4565-b84f-4d994a978624" containerName="maas-api"
Sep 19 01:39:06.719242 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.719079    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca421c49-d4f8-477c-8974-70ffc05b9468" containerName="payload-pre-processing"
Sep 19 01:39:06.719242 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.719088    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c834b565-3fe9-4da0-8f6f-608c0f015c5a" containerName="maas-api"
Sep 19 01:39:06.719242 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.719093    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cac50260-ce71-46b8-b973-91bc32267314" containerName="test-curl-7506-b334e2"
Sep 19 01:39:06.719242 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.719099    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c7bac26-42b5-4b3d-9a03-8205d2691f96" containerName="main"
Sep 19 01:39:06.719242 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.719105    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="33f30bb4-e89e-4c99-9801-787321b0a13d" containerName="main"
Sep 19 01:39:06.719242 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.719110    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c0d4ade-2a79-4833-8c42-306dd75927c6" containerName="maas-api"
Sep 19 01:39:06.722436 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.722421    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7506-e8c0be"
Sep 19 01:39:06.725555 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.725531    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-frkf9\""
Sep 19 01:39:06.728357 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.728334    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7506-e8c0be"]
Sep 19 01:39:06.855553 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.855523    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f7ls\" (UniqueName: \"kubernetes.io/projected/981795d1-6adc-4268-9bf6-c07aa80b89d8-kube-api-access-6f7ls\") pod \"test-curl-7506-e8c0be\" (UID: \"981795d1-6adc-4268-9bf6-c07aa80b89d8\") " pod="opendatahub/test-curl-7506-e8c0be"
Sep 19 01:39:06.956634 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.956601    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f7ls\" (UniqueName: \"kubernetes.io/projected/981795d1-6adc-4268-9bf6-c07aa80b89d8-kube-api-access-6f7ls\") pod \"test-curl-7506-e8c0be\" (UID: \"981795d1-6adc-4268-9bf6-c07aa80b89d8\") " pod="opendatahub/test-curl-7506-e8c0be"
Sep 19 01:39:06.964411 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:06.964379    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f7ls\" (UniqueName: \"kubernetes.io/projected/981795d1-6adc-4268-9bf6-c07aa80b89d8-kube-api-access-6f7ls\") pod \"test-curl-7506-e8c0be\" (UID: \"981795d1-6adc-4268-9bf6-c07aa80b89d8\") " pod="opendatahub/test-curl-7506-e8c0be"
Sep 19 01:39:07.034168 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:07.034133    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7506-e8c0be"
Sep 19 01:39:07.304887 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:07.304860    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7506-e8c0be"]
Sep 19 01:39:07.306681 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:39:07.306650    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod981795d1_6adc_4268_9bf6_c07aa80b89d8.slice/crio-7651d4870d90af1d1ba3d2304e880891e2e422063034e277e3dad6badf3d756b WatchSource:0}: Error finding container 7651d4870d90af1d1ba3d2304e880891e2e422063034e277e3dad6badf3d756b: Status 404 returned error can't find the container with id 7651d4870d90af1d1ba3d2304e880891e2e422063034e277e3dad6badf3d756b
Sep 19 01:39:08.320491 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:08.320447    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7506-e8c0be" event={"ID":"981795d1-6adc-4268-9bf6-c07aa80b89d8","Type":"ContainerStarted","Data":"3da1ac6ffc61ce20abcfa76cce5ef66ecf0a0ca717b20a195b7340beed742147"}
Sep 19 01:39:08.320898 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:08.320519    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7506-e8c0be" event={"ID":"981795d1-6adc-4268-9bf6-c07aa80b89d8","Type":"ContainerStarted","Data":"7651d4870d90af1d1ba3d2304e880891e2e422063034e277e3dad6badf3d756b"}
Sep 19 01:39:08.341537 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:08.341478    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7506-e8c0be" podStartSLOduration=2.341462868 podStartE2EDuration="2.341462868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:39:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:39:08.338211171 +0000 UTC m=+1603.245073244" watchObservedRunningTime="2026-09-19 01:39:08.341462868 +0000 UTC m=+1603.248324916"
Sep 19 01:39:08.959446 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:08.959412    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7506-e8c0be"]
Sep 19 01:39:08.970001 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:08.969972    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7506-e8c0be"]
Sep 19 01:39:09.090686 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:09.090616    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7506-a5ac3d"]
Sep 19 01:39:09.091073 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:09.091031    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="981795d1-6adc-4268-9bf6-c07aa80b89d8" containerName="test-curl-7506-e8c0be"
Sep 19 01:39:09.091073 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:09.091045    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="981795d1-6adc-4268-9bf6-c07aa80b89d8" containerName="test-curl-7506-e8c0be"
Sep 19 01:39:09.091173 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:09.091127    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="981795d1-6adc-4268-9bf6-c07aa80b89d8" containerName="test-curl-7506-e8c0be"
Sep 19 01:39:09.094362 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:09.094344    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7506-a5ac3d"
Sep 19 01:39:09.098167 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:09.098139    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7506-a5ac3d"]
Sep 19 01:39:09.177538 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:09.177480    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksksz\" (UniqueName: \"kubernetes.io/projected/b9ffb495-cfcb-464c-bb41-fe7a6f26bd25-kube-api-access-ksksz\") pod \"test-curl-7506-a5ac3d\" (UID: \"b9ffb495-cfcb-464c-bb41-fe7a6f26bd25\") " pod="opendatahub/test-curl-7506-a5ac3d"
Sep 19 01:39:09.278769 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:09.278737    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksksz\" (UniqueName: \"kubernetes.io/projected/b9ffb495-cfcb-464c-bb41-fe7a6f26bd25-kube-api-access-ksksz\") pod \"test-curl-7506-a5ac3d\" (UID: \"b9ffb495-cfcb-464c-bb41-fe7a6f26bd25\") " pod="opendatahub/test-curl-7506-a5ac3d"
Sep 19 01:39:09.286697 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:09.286675    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksksz\" (UniqueName: \"kubernetes.io/projected/b9ffb495-cfcb-464c-bb41-fe7a6f26bd25-kube-api-access-ksksz\") pod \"test-curl-7506-a5ac3d\" (UID: \"b9ffb495-cfcb-464c-bb41-fe7a6f26bd25\") " pod="opendatahub/test-curl-7506-a5ac3d"
Sep 19 01:39:09.327923 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:39:09.327893    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7506-e8c0be" container="test-curl-7506-e8c0be"
Sep 19 01:39:09.405032 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:09.404962    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7506-a5ac3d"
Sep 19 01:39:09.677544 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:09.677513    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7506-a5ac3d"]
Sep 19 01:39:09.679214 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:39:09.679185    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9ffb495_cfcb_464c_bb41_fe7a6f26bd25.slice/crio-768b530821bb25e3780e58dde9d7ed4f52640769c947777fc969c7cdb0963630 WatchSource:0}: Error finding container 768b530821bb25e3780e58dde9d7ed4f52640769c947777fc969c7cdb0963630: Status 404 returned error can't find the container with id 768b530821bb25e3780e58dde9d7ed4f52640769c947777fc969c7cdb0963630
Sep 19 01:39:10.336412 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:10.336370    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7506-a5ac3d" event={"ID":"b9ffb495-cfcb-464c-bb41-fe7a6f26bd25","Type":"ContainerStarted","Data":"ae5b4af677c3a41e5012977a9485fbc8f09aa0b3365b1d359b80446ad5879b6f"}
Sep 19 01:39:10.336886 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:10.336426    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7506-a5ac3d" event={"ID":"b9ffb495-cfcb-464c-bb41-fe7a6f26bd25","Type":"ContainerStarted","Data":"768b530821bb25e3780e58dde9d7ed4f52640769c947777fc969c7cdb0963630"}
Sep 19 01:39:10.336886 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:10.336456    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7506-e8c0be" podUID="981795d1-6adc-4268-9bf6-c07aa80b89d8" containerName="test-curl-7506-e8c0be" containerID="cri-o://3da1ac6ffc61ce20abcfa76cce5ef66ecf0a0ca717b20a195b7340beed742147" gracePeriod=2
Sep 19 01:39:10.357572 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:10.357529    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7506-a5ac3d" podStartSLOduration=1.35751537 podStartE2EDuration="1.35751537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:39:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:39:10.355772343 +0000 UTC m=+1605.262634390" watchObservedRunningTime="2026-09-19 01:39:10.35751537 +0000 UTC m=+1605.264377442"
Sep 19 01:39:10.949402 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:10.949359    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7506-a5ac3d"]
Sep 19 01:39:10.953231 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:10.953203    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7506-a5ac3d"]
Sep 19 01:39:11.343149 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:39:11.343122    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7506-a5ac3d" container="test-curl-7506-a5ac3d"
Sep 19 01:39:12.349681 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:12.349644    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7506-a5ac3d" podUID="b9ffb495-cfcb-464c-bb41-fe7a6f26bd25" containerName="test-curl-7506-a5ac3d" containerID="cri-o://ae5b4af677c3a41e5012977a9485fbc8f09aa0b3365b1d359b80446ad5879b6f" gracePeriod=2
Sep 19 01:39:12.529215 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:12.529185    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7506-e8c0be"
Sep 19 01:39:12.608825 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:12.608745    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6f7ls\" (UniqueName: \"kubernetes.io/projected/981795d1-6adc-4268-9bf6-c07aa80b89d8-kube-api-access-6f7ls\") pod \"981795d1-6adc-4268-9bf6-c07aa80b89d8\" (UID: \"981795d1-6adc-4268-9bf6-c07aa80b89d8\") "
Sep 19 01:39:12.610832 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:12.610806    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/981795d1-6adc-4268-9bf6-c07aa80b89d8-kube-api-access-6f7ls" (OuterVolumeSpecName: "kube-api-access-6f7ls") pod "981795d1-6adc-4268-9bf6-c07aa80b89d8" (UID: "981795d1-6adc-4268-9bf6-c07aa80b89d8"). InnerVolumeSpecName "kube-api-access-6f7ls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:39:12.710260 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:12.710227    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6f7ls\" (UniqueName: \"kubernetes.io/projected/981795d1-6adc-4268-9bf6-c07aa80b89d8-kube-api-access-6f7ls\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:39:13.068830 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:13.068790    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="981795d1-6adc-4268-9bf6-c07aa80b89d8" path="/var/lib/kubelet/pods/981795d1-6adc-4268-9bf6-c07aa80b89d8/volumes"
Sep 19 01:39:13.356726 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:13.356639    2580 generic.go:356] "Generic (PLEG): container finished" podID="981795d1-6adc-4268-9bf6-c07aa80b89d8" containerID="3da1ac6ffc61ce20abcfa76cce5ef66ecf0a0ca717b20a195b7340beed742147" exitCode=137
Sep 19 01:39:13.356726 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:13.356681    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7506-e8c0be"
Sep 19 01:39:13.357140 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:13.356733    2580 scope.go:117] "RemoveContainer" containerID="3da1ac6ffc61ce20abcfa76cce5ef66ecf0a0ca717b20a195b7340beed742147"
Sep 19 01:39:13.365950 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:13.365931    2580 scope.go:117] "RemoveContainer" containerID="3da1ac6ffc61ce20abcfa76cce5ef66ecf0a0ca717b20a195b7340beed742147"
Sep 19 01:39:13.366237 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:39:13.366219    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3da1ac6ffc61ce20abcfa76cce5ef66ecf0a0ca717b20a195b7340beed742147\": container with ID starting with 3da1ac6ffc61ce20abcfa76cce5ef66ecf0a0ca717b20a195b7340beed742147 not found: ID does not exist" containerID="3da1ac6ffc61ce20abcfa76cce5ef66ecf0a0ca717b20a195b7340beed742147"
Sep 19 01:39:13.366282 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:13.366249    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3da1ac6ffc61ce20abcfa76cce5ef66ecf0a0ca717b20a195b7340beed742147"} err="failed to get container status \"3da1ac6ffc61ce20abcfa76cce5ef66ecf0a0ca717b20a195b7340beed742147\": rpc error: code = NotFound desc = could not find container \"3da1ac6ffc61ce20abcfa76cce5ef66ecf0a0ca717b20a195b7340beed742147\": container with ID starting with 3da1ac6ffc61ce20abcfa76cce5ef66ecf0a0ca717b20a195b7340beed742147 not found: ID does not exist"
Sep 19 01:39:14.521920 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:14.521896    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7506-a5ac3d"
Sep 19 01:39:14.628702 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:14.628627    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ksksz\" (UniqueName: \"kubernetes.io/projected/b9ffb495-cfcb-464c-bb41-fe7a6f26bd25-kube-api-access-ksksz\") pod \"b9ffb495-cfcb-464c-bb41-fe7a6f26bd25\" (UID: \"b9ffb495-cfcb-464c-bb41-fe7a6f26bd25\") "
Sep 19 01:39:14.630766 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:14.630739    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9ffb495-cfcb-464c-bb41-fe7a6f26bd25-kube-api-access-ksksz" (OuterVolumeSpecName: "kube-api-access-ksksz") pod "b9ffb495-cfcb-464c-bb41-fe7a6f26bd25" (UID: "b9ffb495-cfcb-464c-bb41-fe7a6f26bd25"). InnerVolumeSpecName "kube-api-access-ksksz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:39:14.729490 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:14.729459    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ksksz\" (UniqueName: \"kubernetes.io/projected/b9ffb495-cfcb-464c-bb41-fe7a6f26bd25-kube-api-access-ksksz\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:39:15.063930 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:15.063901    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9ffb495-cfcb-464c-bb41-fe7a6f26bd25" path="/var/lib/kubelet/pods/b9ffb495-cfcb-464c-bb41-fe7a6f26bd25/volumes"
Sep 19 01:39:15.371343 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:15.371253    2580 generic.go:356] "Generic (PLEG): container finished" podID="b9ffb495-cfcb-464c-bb41-fe7a6f26bd25" containerID="ae5b4af677c3a41e5012977a9485fbc8f09aa0b3365b1d359b80446ad5879b6f" exitCode=137
Sep 19 01:39:15.371343 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:15.371296    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7506-a5ac3d"
Sep 19 01:39:15.371588 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:15.371346    2580 scope.go:117] "RemoveContainer" containerID="ae5b4af677c3a41e5012977a9485fbc8f09aa0b3365b1d359b80446ad5879b6f"
Sep 19 01:39:15.380576 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:15.380357    2580 scope.go:117] "RemoveContainer" containerID="ae5b4af677c3a41e5012977a9485fbc8f09aa0b3365b1d359b80446ad5879b6f"
Sep 19 01:39:15.380644 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:39:15.380627    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae5b4af677c3a41e5012977a9485fbc8f09aa0b3365b1d359b80446ad5879b6f\": container with ID starting with ae5b4af677c3a41e5012977a9485fbc8f09aa0b3365b1d359b80446ad5879b6f not found: ID does not exist" containerID="ae5b4af677c3a41e5012977a9485fbc8f09aa0b3365b1d359b80446ad5879b6f"
Sep 19 01:39:15.380685 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:39:15.380647    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae5b4af677c3a41e5012977a9485fbc8f09aa0b3365b1d359b80446ad5879b6f"} err="failed to get container status \"ae5b4af677c3a41e5012977a9485fbc8f09aa0b3365b1d359b80446ad5879b6f\": rpc error: code = NotFound desc = could not find container \"ae5b4af677c3a41e5012977a9485fbc8f09aa0b3365b1d359b80446ad5879b6f\": container with ID starting with ae5b4af677c3a41e5012977a9485fbc8f09aa0b3365b1d359b80446ad5879b6f not found: ID does not exist"
Sep 19 01:40:28.919758 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:28.919718    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d"]
Sep 19 01:40:28.920548 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:28.920527    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9ffb495-cfcb-464c-bb41-fe7a6f26bd25" containerName="test-curl-7506-a5ac3d"
Sep 19 01:40:28.920548 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:28.920549    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9ffb495-cfcb-464c-bb41-fe7a6f26bd25" containerName="test-curl-7506-a5ac3d"
Sep 19 01:40:28.920677 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:28.920658    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9ffb495-cfcb-464c-bb41-fe7a6f26bd25" containerName="test-curl-7506-a5ac3d"
Sep 19 01:40:28.923886 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:28.923867    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d"
Sep 19 01:40:28.926247 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:28.926223    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-a4edd7-serving-cert\""
Sep 19 01:40:28.934725 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:28.934484    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d"]
Sep 19 01:40:28.988703 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:28.988674    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4858af12-7b70-427b-b689-6a6c87f992fe-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d\" (UID: \"4858af12-7b70-427b-b689-6a6c87f992fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d"
Sep 19 01:40:28.988887 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:28.988717    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4858af12-7b70-427b-b689-6a6c87f992fe-metrics-certs\") pod \"maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d\" (UID: \"4858af12-7b70-427b-b689-6a6c87f992fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d"
Sep 19 01:40:28.988887 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:28.988739    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82xgb\" (UniqueName: \"kubernetes.io/projected/4858af12-7b70-427b-b689-6a6c87f992fe-kube-api-access-82xgb\") pod \"maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d\" (UID: \"4858af12-7b70-427b-b689-6a6c87f992fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d"
Sep 19 01:40:28.988887 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:28.988838    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4858af12-7b70-427b-b689-6a6c87f992fe-maas-api-tls\") pod \"maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d\" (UID: \"4858af12-7b70-427b-b689-6a6c87f992fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d"
Sep 19 01:40:29.089726 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:29.089693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4858af12-7b70-427b-b689-6a6c87f992fe-maas-api-tls\") pod \"maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d\" (UID: \"4858af12-7b70-427b-b689-6a6c87f992fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d"
Sep 19 01:40:29.089908 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:29.089795    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4858af12-7b70-427b-b689-6a6c87f992fe-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d\" (UID: \"4858af12-7b70-427b-b689-6a6c87f992fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d"
Sep 19 01:40:29.089908 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:29.089851    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4858af12-7b70-427b-b689-6a6c87f992fe-metrics-certs\") pod \"maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d\" (UID: \"4858af12-7b70-427b-b689-6a6c87f992fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d"
Sep 19 01:40:29.090052 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:29.089978    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82xgb\" (UniqueName: \"kubernetes.io/projected/4858af12-7b70-427b-b689-6a6c87f992fe-kube-api-access-82xgb\") pod \"maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d\" (UID: \"4858af12-7b70-427b-b689-6a6c87f992fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d"
Sep 19 01:40:29.090661 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:29.090637    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4858af12-7b70-427b-b689-6a6c87f992fe-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d\" (UID: \"4858af12-7b70-427b-b689-6a6c87f992fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d"
Sep 19 01:40:29.092308 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:29.092278    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4858af12-7b70-427b-b689-6a6c87f992fe-metrics-certs\") pod \"maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d\" (UID: \"4858af12-7b70-427b-b689-6a6c87f992fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d"
Sep 19 01:40:29.092423 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:29.092342    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4858af12-7b70-427b-b689-6a6c87f992fe-maas-api-tls\") pod \"maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d\" (UID: \"4858af12-7b70-427b-b689-6a6c87f992fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d"
Sep 19 01:40:29.099580 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:29.099557    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82xgb\" (UniqueName: \"kubernetes.io/projected/4858af12-7b70-427b-b689-6a6c87f992fe-kube-api-access-82xgb\") pod \"maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d\" (UID: \"4858af12-7b70-427b-b689-6a6c87f992fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d"
Sep 19 01:40:29.239662 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:29.239619    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d"
Sep 19 01:40:29.548996 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:29.548961    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d"]
Sep 19 01:40:29.549127 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:40:29.548973    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4858af12_7b70_427b_b689_6a6c87f992fe.slice/crio-2e49e05003934e20932759424da66467b521d281eed243c609dff28bfa30609e WatchSource:0}: Error finding container 2e49e05003934e20932759424da66467b521d281eed243c609dff28bfa30609e: Status 404 returned error can't find the container with id 2e49e05003934e20932759424da66467b521d281eed243c609dff28bfa30609e
Sep 19 01:40:29.827632 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:29.827534    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d" event={"ID":"4858af12-7b70-427b-b689-6a6c87f992fe","Type":"ContainerStarted","Data":"2e49e05003934e20932759424da66467b521d281eed243c609dff28bfa30609e"}
Sep 19 01:40:30.838290 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:30.838246    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d" event={"ID":"4858af12-7b70-427b-b689-6a6c87f992fe","Type":"ContainerStarted","Data":"2a951e031daf17f3faebe36ac8bb0b5373adb3468a9212fb37284731780ffb2d"}
Sep 19 01:40:30.838794 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:30.838333    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d"
Sep 19 01:40:30.858547 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:30.858487    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d" podStartSLOduration=2.058597582 podStartE2EDuration="2.858471661s" totalImagesPullingTime="799.874079ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:40:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:40:30.856268836 +0000 UTC m=+1685.763130883" watchObservedRunningTime="2026-09-19 01:40:30.858471661 +0000 UTC m=+1685.765333743"
Sep 19 01:40:36.850676 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:40:36.850645    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d"
Sep 19 01:41:03.654154 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:41:03.654122    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn"]
Sep 19 01:41:03.658431 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:41:03.658406    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn"
Sep 19 01:41:03.660614 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:41:03.660587    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-14de36-serving-cert\""
Sep 19 01:41:03.672374 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:41:03.672348    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn"]
Sep 19 01:41:03.725112 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:41:03.725078    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65-metrics-certs\") pod \"maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn\" (UID: \"c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn"
Sep 19 01:41:03.725112 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:41:03.725111    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhv9p\" (UniqueName: \"kubernetes.io/projected/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65-kube-api-access-qhv9p\") pod \"maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn\" (UID: \"c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn"
Sep 19 01:41:03.725328 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:41:03.725218    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65-maas-api-tls\") pod \"maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn\" (UID: \"c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn"
Sep 19 01:41:03.725328 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:41:03.725262    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn\" (UID: \"c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn"
Sep 19 01:41:03.826358 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:41:03.826326    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn\" (UID: \"c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn"
Sep 19 01:41:03.826574 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:41:03.826382    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65-metrics-certs\") pod \"maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn\" (UID: \"c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn"
Sep 19 01:41:03.826574 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:41:03.826403    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhv9p\" (UniqueName: \"kubernetes.io/projected/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65-kube-api-access-qhv9p\") pod \"maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn\" (UID: \"c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn"
Sep 19 01:41:03.826574 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:41:03.826469    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65-maas-api-tls\") pod \"maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn\" (UID: \"c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn"
Sep 19 01:41:03.827055 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:41:03.827027    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn\" (UID: \"c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn"
Sep 19 01:41:03.828924 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:41:03.828902    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65-maas-api-tls\") pod \"maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn\" (UID: \"c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn"
Sep 19 01:41:03.829053 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:41:03.829034    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65-metrics-certs\") pod \"maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn\" (UID: \"c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn"
Sep 19 01:41:03.835235 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:41:03.835215    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhv9p\" (UniqueName: \"kubernetes.io/projected/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65-kube-api-access-qhv9p\") pod \"maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn\" (UID: \"c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn"
Sep 19 01:41:03.971246 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:41:03.971155    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn"
Sep 19 01:41:04.253295 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:41:04.253256    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn"]
Sep 19 01:41:04.255971 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:41:04.255946    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9bc2cdd_75b9_44d0_90c3_a7ff6b4c6d65.slice/crio-bb003e665b772a8529bb1d74e97c078488d0e76a86249774217fa958e861dae7 WatchSource:0}: Error finding container bb003e665b772a8529bb1d74e97c078488d0e76a86249774217fa958e861dae7: Status 404 returned error can't find the container with id bb003e665b772a8529bb1d74e97c078488d0e76a86249774217fa958e861dae7
Sep 19 01:41:05.056379 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:41:05.056338    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn" event={"ID":"c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65","Type":"ContainerStarted","Data":"d395822449052d6bdc99d9cffd6ecfb087667ea636d02d0ee2bf998abda447df"}
Sep 19 01:41:05.056952 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:41:05.056435    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn" event={"ID":"c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65","Type":"ContainerStarted","Data":"bb003e665b772a8529bb1d74e97c078488d0e76a86249774217fa958e861dae7"}
Sep 19 01:41:05.063839 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:41:05.063813    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn"
Sep 19 01:41:05.077156 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:41:05.077110    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn" podStartSLOduration=1.793447628 podStartE2EDuration="2.077098523s" totalImagesPullingTime="283.650895ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:41:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:41:05.076386163 +0000 UTC m=+1719.983248236" watchObservedRunningTime="2026-09-19 01:41:05.077098523 +0000 UTC m=+1719.983960571"
Sep 19 01:41:12.073152 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:41:12.073120    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn"
Sep 19 01:41:36.270720 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:41:36.270693    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-42wkt_c9ba5520-238c-4d7b-9266-16a1c0f76e8d/payload-processing/0.log"
Sep 19 01:42:20.100214 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:20.100184    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-42wkt_c9ba5520-238c-4d7b-9266-16a1c0f76e8d/payload-processing/0.log"
Sep 19 01:42:27.262015 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:27.261982    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f6bdh_74c8e2f6-1b01-4bd1-b491-ad407a15a109/console-operator/1.log"
Sep 19 01:42:27.268227 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:27.268203    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f6bdh_74c8e2f6-1b01-4bd1-b491-ad407a15a109/console-operator/1.log"
Sep 19 01:42:27.719940 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:27.719851    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn"]
Sep 19 01:42:27.723168 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:27.723146    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn"
Sep 19 01:42:27.725713 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:27.725691    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-df8e58-serving-cert\""
Sep 19 01:42:27.736517 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:27.735949    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn"]
Sep 19 01:42:27.819267 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:27.819233    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31c799a2-1a45-4af6-b751-993ca034a4bf-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn\" (UID: \"31c799a2-1a45-4af6-b751-993ca034a4bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn"
Sep 19 01:42:27.819442 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:27.819276    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31c799a2-1a45-4af6-b751-993ca034a4bf-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn\" (UID: \"31c799a2-1a45-4af6-b751-993ca034a4bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn"
Sep 19 01:42:27.819442 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:27.819346    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrfbw\" (UniqueName: \"kubernetes.io/projected/31c799a2-1a45-4af6-b751-993ca034a4bf-kube-api-access-hrfbw\") pod \"maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn\" (UID: \"31c799a2-1a45-4af6-b751-993ca034a4bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn"
Sep 19 01:42:27.819442 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:27.819372    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31c799a2-1a45-4af6-b751-993ca034a4bf-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn\" (UID: \"31c799a2-1a45-4af6-b751-993ca034a4bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn"
Sep 19 01:42:27.920135 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:27.920103    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31c799a2-1a45-4af6-b751-993ca034a4bf-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn\" (UID: \"31c799a2-1a45-4af6-b751-993ca034a4bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn"
Sep 19 01:42:27.920302 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:27.920154    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31c799a2-1a45-4af6-b751-993ca034a4bf-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn\" (UID: \"31c799a2-1a45-4af6-b751-993ca034a4bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn"
Sep 19 01:42:27.920302 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:27.920214    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrfbw\" (UniqueName: \"kubernetes.io/projected/31c799a2-1a45-4af6-b751-993ca034a4bf-kube-api-access-hrfbw\") pod \"maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn\" (UID: \"31c799a2-1a45-4af6-b751-993ca034a4bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn"
Sep 19 01:42:27.920302 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:27.920242    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31c799a2-1a45-4af6-b751-993ca034a4bf-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn\" (UID: \"31c799a2-1a45-4af6-b751-993ca034a4bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn"
Sep 19 01:42:27.920841 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:27.920814    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31c799a2-1a45-4af6-b751-993ca034a4bf-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn\" (UID: \"31c799a2-1a45-4af6-b751-993ca034a4bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn"
Sep 19 01:42:27.923123 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:27.923083    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31c799a2-1a45-4af6-b751-993ca034a4bf-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn\" (UID: \"31c799a2-1a45-4af6-b751-993ca034a4bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn"
Sep 19 01:42:27.923246 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:27.923125    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31c799a2-1a45-4af6-b751-993ca034a4bf-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn\" (UID: \"31c799a2-1a45-4af6-b751-993ca034a4bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn"
Sep 19 01:42:27.929905 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:27.929858    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrfbw\" (UniqueName: \"kubernetes.io/projected/31c799a2-1a45-4af6-b751-993ca034a4bf-kube-api-access-hrfbw\") pod \"maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn\" (UID: \"31c799a2-1a45-4af6-b751-993ca034a4bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn"
Sep 19 01:42:28.039220 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:28.039185    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn"
Sep 19 01:42:28.318666 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:28.318639    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn"]
Sep 19 01:42:28.320013 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:42:28.319981    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31c799a2_1a45_4af6_b751_993ca034a4bf.slice/crio-0d9acf2576f15f4d086ba3592cb3521ad6d3fde2d295abb8b4f377bf6f8cfd06 WatchSource:0}: Error finding container 0d9acf2576f15f4d086ba3592cb3521ad6d3fde2d295abb8b4f377bf6f8cfd06: Status 404 returned error can't find the container with id 0d9acf2576f15f4d086ba3592cb3521ad6d3fde2d295abb8b4f377bf6f8cfd06
Sep 19 01:42:28.321626 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:28.321606    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 01:42:28.587441 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:28.587356    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn" event={"ID":"31c799a2-1a45-4af6-b751-993ca034a4bf","Type":"ContainerStarted","Data":"0d9acf2576f15f4d086ba3592cb3521ad6d3fde2d295abb8b4f377bf6f8cfd06"}
Sep 19 01:42:29.594864 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:29.594824    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn" event={"ID":"31c799a2-1a45-4af6-b751-993ca034a4bf","Type":"ContainerStarted","Data":"9c7a20b2bba54798588b77a4d5fcf539cdb9bae89d8837b7a22952553493457b"}
Sep 19 01:42:29.595247 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:29.594886    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn"
Sep 19 01:42:29.619358 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:29.619312    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn" podStartSLOduration=2.207727406 podStartE2EDuration="2.619295519s" totalImagesPullingTime="411.568113ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:42:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:42:29.616744215 +0000 UTC m=+1804.523606264" watchObservedRunningTime="2026-09-19 01:42:29.619295519 +0000 UTC m=+1804.526157566"
Sep 19 01:42:35.607292 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:42:35.607262    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn"
Sep 19 01:43:06.019657 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:06.019151    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn"]
Sep 19 01:43:06.019657 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:06.019439    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn" podUID="31c799a2-1a45-4af6-b751-993ca034a4bf" containerName="maas-api" containerID="cri-o://9c7a20b2bba54798588b77a4d5fcf539cdb9bae89d8837b7a22952553493457b" gracePeriod=30
Sep 19 01:43:06.278548 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:06.278480    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn"
Sep 19 01:43:06.372480 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:06.372446    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31c799a2-1a45-4af6-b751-993ca034a4bf-metrics-certs\") pod \"31c799a2-1a45-4af6-b751-993ca034a4bf\" (UID: \"31c799a2-1a45-4af6-b751-993ca034a4bf\") "
Sep 19 01:43:06.372480 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:06.372487    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31c799a2-1a45-4af6-b751-993ca034a4bf-maas-api-tls\") pod \"31c799a2-1a45-4af6-b751-993ca034a4bf\" (UID: \"31c799a2-1a45-4af6-b751-993ca034a4bf\") "
Sep 19 01:43:06.372759 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:06.372563    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31c799a2-1a45-4af6-b751-993ca034a4bf-openshift-service-ca\") pod \"31c799a2-1a45-4af6-b751-993ca034a4bf\" (UID: \"31c799a2-1a45-4af6-b751-993ca034a4bf\") "
Sep 19 01:43:06.372759 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:06.372617    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrfbw\" (UniqueName: \"kubernetes.io/projected/31c799a2-1a45-4af6-b751-993ca034a4bf-kube-api-access-hrfbw\") pod \"31c799a2-1a45-4af6-b751-993ca034a4bf\" (UID: \"31c799a2-1a45-4af6-b751-993ca034a4bf\") "
Sep 19 01:43:06.373094 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:06.373053    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31c799a2-1a45-4af6-b751-993ca034a4bf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "31c799a2-1a45-4af6-b751-993ca034a4bf" (UID: "31c799a2-1a45-4af6-b751-993ca034a4bf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:43:06.375179 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:06.375149    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31c799a2-1a45-4af6-b751-993ca034a4bf-kube-api-access-hrfbw" (OuterVolumeSpecName: "kube-api-access-hrfbw") pod "31c799a2-1a45-4af6-b751-993ca034a4bf" (UID: "31c799a2-1a45-4af6-b751-993ca034a4bf"). InnerVolumeSpecName "kube-api-access-hrfbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:43:06.375291 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:06.375242    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31c799a2-1a45-4af6-b751-993ca034a4bf-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "31c799a2-1a45-4af6-b751-993ca034a4bf" (UID: "31c799a2-1a45-4af6-b751-993ca034a4bf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:43:06.375705 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:06.375676    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31c799a2-1a45-4af6-b751-993ca034a4bf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "31c799a2-1a45-4af6-b751-993ca034a4bf" (UID: "31c799a2-1a45-4af6-b751-993ca034a4bf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:43:06.474009 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:06.473977    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrfbw\" (UniqueName: \"kubernetes.io/projected/31c799a2-1a45-4af6-b751-993ca034a4bf-kube-api-access-hrfbw\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:43:06.474009 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:06.474005    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31c799a2-1a45-4af6-b751-993ca034a4bf-metrics-certs\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:43:06.474009 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:06.474014    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31c799a2-1a45-4af6-b751-993ca034a4bf-maas-api-tls\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:43:06.474238 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:06.474023    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31c799a2-1a45-4af6-b751-993ca034a4bf-openshift-service-ca\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:43:06.821602 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:06.821557    2580 generic.go:356] "Generic (PLEG): container finished" podID="31c799a2-1a45-4af6-b751-993ca034a4bf" containerID="9c7a20b2bba54798588b77a4d5fcf539cdb9bae89d8837b7a22952553493457b" exitCode=0
Sep 19 01:43:06.821764 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:06.821605    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn"
Sep 19 01:43:06.821764 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:06.821632    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn" event={"ID":"31c799a2-1a45-4af6-b751-993ca034a4bf","Type":"ContainerDied","Data":"9c7a20b2bba54798588b77a4d5fcf539cdb9bae89d8837b7a22952553493457b"}
Sep 19 01:43:06.821764 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:06.821694    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn" event={"ID":"31c799a2-1a45-4af6-b751-993ca034a4bf","Type":"ContainerDied","Data":"0d9acf2576f15f4d086ba3592cb3521ad6d3fde2d295abb8b4f377bf6f8cfd06"}
Sep 19 01:43:06.821764 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:06.821725    2580 scope.go:117] "RemoveContainer" containerID="9c7a20b2bba54798588b77a4d5fcf539cdb9bae89d8837b7a22952553493457b"
Sep 19 01:43:06.832820 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:06.832802    2580 scope.go:117] "RemoveContainer" containerID="9c7a20b2bba54798588b77a4d5fcf539cdb9bae89d8837b7a22952553493457b"
Sep 19 01:43:06.833126 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:43:06.833108    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c7a20b2bba54798588b77a4d5fcf539cdb9bae89d8837b7a22952553493457b\": container with ID starting with 9c7a20b2bba54798588b77a4d5fcf539cdb9bae89d8837b7a22952553493457b not found: ID does not exist" containerID="9c7a20b2bba54798588b77a4d5fcf539cdb9bae89d8837b7a22952553493457b"
Sep 19 01:43:06.833193 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:06.833140    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c7a20b2bba54798588b77a4d5fcf539cdb9bae89d8837b7a22952553493457b"} err="failed to get container status \"9c7a20b2bba54798588b77a4d5fcf539cdb9bae89d8837b7a22952553493457b\": rpc error: code = NotFound desc = could not find container \"9c7a20b2bba54798588b77a4d5fcf539cdb9bae89d8837b7a22952553493457b\": container with ID starting with 9c7a20b2bba54798588b77a4d5fcf539cdb9bae89d8837b7a22952553493457b not found: ID does not exist"
Sep 19 01:43:06.844357 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:06.844334    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn"]
Sep 19 01:43:06.850254 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:06.850233    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-df8e58-5c4bcf665c-ckbdn"]
Sep 19 01:43:07.065093 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:07.065061    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31c799a2-1a45-4af6-b751-993ca034a4bf" path="/var/lib/kubelet/pods/31c799a2-1a45-4af6-b751-993ca034a4bf/volumes"
Sep 19 01:43:33.188622 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:33.188589    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d"]
Sep 19 01:43:33.189098 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:33.188813    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d" podUID="4858af12-7b70-427b-b689-6a6c87f992fe" containerName="maas-api" containerID="cri-o://2a951e031daf17f3faebe36ac8bb0b5373adb3468a9212fb37284731780ffb2d" gracePeriod=30
Sep 19 01:43:33.469602 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:33.469566    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d"
Sep 19 01:43:33.528999 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:33.528971    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4858af12-7b70-427b-b689-6a6c87f992fe-metrics-certs\") pod \"4858af12-7b70-427b-b689-6a6c87f992fe\" (UID: \"4858af12-7b70-427b-b689-6a6c87f992fe\") "
Sep 19 01:43:33.529183 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:33.529084    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4858af12-7b70-427b-b689-6a6c87f992fe-maas-api-tls\") pod \"4858af12-7b70-427b-b689-6a6c87f992fe\" (UID: \"4858af12-7b70-427b-b689-6a6c87f992fe\") "
Sep 19 01:43:33.529183 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:33.529113    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4858af12-7b70-427b-b689-6a6c87f992fe-openshift-service-ca\") pod \"4858af12-7b70-427b-b689-6a6c87f992fe\" (UID: \"4858af12-7b70-427b-b689-6a6c87f992fe\") "
Sep 19 01:43:33.529300 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:33.529202    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82xgb\" (UniqueName: \"kubernetes.io/projected/4858af12-7b70-427b-b689-6a6c87f992fe-kube-api-access-82xgb\") pod \"4858af12-7b70-427b-b689-6a6c87f992fe\" (UID: \"4858af12-7b70-427b-b689-6a6c87f992fe\") "
Sep 19 01:43:33.530027 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:33.529997    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4858af12-7b70-427b-b689-6a6c87f992fe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4858af12-7b70-427b-b689-6a6c87f992fe" (UID: "4858af12-7b70-427b-b689-6a6c87f992fe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:43:33.531886 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:33.531860    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4858af12-7b70-427b-b689-6a6c87f992fe-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4858af12-7b70-427b-b689-6a6c87f992fe" (UID: "4858af12-7b70-427b-b689-6a6c87f992fe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:43:33.532140 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:33.532080    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4858af12-7b70-427b-b689-6a6c87f992fe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4858af12-7b70-427b-b689-6a6c87f992fe" (UID: "4858af12-7b70-427b-b689-6a6c87f992fe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:43:33.532140 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:33.532085    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4858af12-7b70-427b-b689-6a6c87f992fe-kube-api-access-82xgb" (OuterVolumeSpecName: "kube-api-access-82xgb") pod "4858af12-7b70-427b-b689-6a6c87f992fe" (UID: "4858af12-7b70-427b-b689-6a6c87f992fe"). InnerVolumeSpecName "kube-api-access-82xgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:43:33.630717 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:33.630684    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4858af12-7b70-427b-b689-6a6c87f992fe-maas-api-tls\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:43:33.630717 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:33.630714    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4858af12-7b70-427b-b689-6a6c87f992fe-openshift-service-ca\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:43:33.630717 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:33.630725    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82xgb\" (UniqueName: \"kubernetes.io/projected/4858af12-7b70-427b-b689-6a6c87f992fe-kube-api-access-82xgb\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:43:33.631022 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:33.630734    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4858af12-7b70-427b-b689-6a6c87f992fe-metrics-certs\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:43:33.991671 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:33.991636    2580 generic.go:356] "Generic (PLEG): container finished" podID="4858af12-7b70-427b-b689-6a6c87f992fe" containerID="2a951e031daf17f3faebe36ac8bb0b5373adb3468a9212fb37284731780ffb2d" exitCode=0
Sep 19 01:43:33.991864 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:33.991691    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d"
Sep 19 01:43:33.991864 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:33.991724    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d" event={"ID":"4858af12-7b70-427b-b689-6a6c87f992fe","Type":"ContainerDied","Data":"2a951e031daf17f3faebe36ac8bb0b5373adb3468a9212fb37284731780ffb2d"}
Sep 19 01:43:33.991864 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:33.991782    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d" event={"ID":"4858af12-7b70-427b-b689-6a6c87f992fe","Type":"ContainerDied","Data":"2e49e05003934e20932759424da66467b521d281eed243c609dff28bfa30609e"}
Sep 19 01:43:33.991864 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:33.991806    2580 scope.go:117] "RemoveContainer" containerID="2a951e031daf17f3faebe36ac8bb0b5373adb3468a9212fb37284731780ffb2d"
Sep 19 01:43:34.002098 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:34.002080    2580 scope.go:117] "RemoveContainer" containerID="2a951e031daf17f3faebe36ac8bb0b5373adb3468a9212fb37284731780ffb2d"
Sep 19 01:43:34.002370 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:43:34.002354    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a951e031daf17f3faebe36ac8bb0b5373adb3468a9212fb37284731780ffb2d\": container with ID starting with 2a951e031daf17f3faebe36ac8bb0b5373adb3468a9212fb37284731780ffb2d not found: ID does not exist" containerID="2a951e031daf17f3faebe36ac8bb0b5373adb3468a9212fb37284731780ffb2d"
Sep 19 01:43:34.002430 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:34.002380    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a951e031daf17f3faebe36ac8bb0b5373adb3468a9212fb37284731780ffb2d"} err="failed to get container status \"2a951e031daf17f3faebe36ac8bb0b5373adb3468a9212fb37284731780ffb2d\": rpc error: code = NotFound desc = could not find container \"2a951e031daf17f3faebe36ac8bb0b5373adb3468a9212fb37284731780ffb2d\": container with ID starting with 2a951e031daf17f3faebe36ac8bb0b5373adb3468a9212fb37284731780ffb2d not found: ID does not exist"
Sep 19 01:43:34.014417 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:34.014388    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d"]
Sep 19 01:43:34.018810 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:34.018788    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4edd7-f99d84cd5-xch7d"]
Sep 19 01:43:35.064053 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:43:35.064019    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4858af12-7b70-427b-b689-6a6c87f992fe" path="/var/lib/kubelet/pods/4858af12-7b70-427b-b689-6a6c87f992fe/volumes"
Sep 19 01:44:39.502266 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:39.502221    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn"]
Sep 19 01:44:39.502824 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:39.502522    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn" podUID="c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65" containerName="maas-api" containerID="cri-o://d395822449052d6bdc99d9cffd6ecfb087667ea636d02d0ee2bf998abda447df" gracePeriod=30
Sep 19 01:44:39.775066 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:39.775032    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn"
Sep 19 01:44:39.938142 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:39.938099    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65-maas-api-tls\") pod \"c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65\" (UID: \"c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65\") "
Sep 19 01:44:39.938405 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:39.938155    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65-metrics-certs\") pod \"c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65\" (UID: \"c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65\") "
Sep 19 01:44:39.938405 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:39.938189    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhv9p\" (UniqueName: \"kubernetes.io/projected/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65-kube-api-access-qhv9p\") pod \"c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65\" (UID: \"c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65\") "
Sep 19 01:44:39.938405 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:39.938218    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65-openshift-service-ca\") pod \"c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65\" (UID: \"c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65\") "
Sep 19 01:44:39.938957 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:39.938927    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65" (UID: "c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:44:39.940774 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:39.940749    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65" (UID: "c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:44:39.940897 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:39.940798    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65-kube-api-access-qhv9p" (OuterVolumeSpecName: "kube-api-access-qhv9p") pod "c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65" (UID: "c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65"). InnerVolumeSpecName "kube-api-access-qhv9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:44:39.941014 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:39.940995    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65" (UID: "c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:44:40.039279 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:40.039189    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65-maas-api-tls\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:44:40.039279 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:40.039228    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65-metrics-certs\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:44:40.039279 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:40.039242    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhv9p\" (UniqueName: \"kubernetes.io/projected/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65-kube-api-access-qhv9p\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:44:40.039279 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:40.039257    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65-openshift-service-ca\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:44:40.394106 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:40.394015    2580 generic.go:356] "Generic (PLEG): container finished" podID="c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65" containerID="d395822449052d6bdc99d9cffd6ecfb087667ea636d02d0ee2bf998abda447df" exitCode=0
Sep 19 01:44:40.394106 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:40.394062    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn"
Sep 19 01:44:40.394106 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:40.394066    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn" event={"ID":"c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65","Type":"ContainerDied","Data":"d395822449052d6bdc99d9cffd6ecfb087667ea636d02d0ee2bf998abda447df"}
Sep 19 01:44:40.394357 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:40.394119    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn" event={"ID":"c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65","Type":"ContainerDied","Data":"bb003e665b772a8529bb1d74e97c078488d0e76a86249774217fa958e861dae7"}
Sep 19 01:44:40.394357 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:40.394150    2580 scope.go:117] "RemoveContainer" containerID="d395822449052d6bdc99d9cffd6ecfb087667ea636d02d0ee2bf998abda447df"
Sep 19 01:44:40.407184 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:40.407159    2580 scope.go:117] "RemoveContainer" containerID="d395822449052d6bdc99d9cffd6ecfb087667ea636d02d0ee2bf998abda447df"
Sep 19 01:44:40.408052 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:44:40.408024    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d395822449052d6bdc99d9cffd6ecfb087667ea636d02d0ee2bf998abda447df\": container with ID starting with d395822449052d6bdc99d9cffd6ecfb087667ea636d02d0ee2bf998abda447df not found: ID does not exist" containerID="d395822449052d6bdc99d9cffd6ecfb087667ea636d02d0ee2bf998abda447df"
Sep 19 01:44:40.408170 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:40.408058    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d395822449052d6bdc99d9cffd6ecfb087667ea636d02d0ee2bf998abda447df"} err="failed to get container status \"d395822449052d6bdc99d9cffd6ecfb087667ea636d02d0ee2bf998abda447df\": rpc error: code = NotFound desc = could not find container \"d395822449052d6bdc99d9cffd6ecfb087667ea636d02d0ee2bf998abda447df\": container with ID starting with d395822449052d6bdc99d9cffd6ecfb087667ea636d02d0ee2bf998abda447df not found: ID does not exist"
Sep 19 01:44:40.421575 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:40.421552    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn"]
Sep 19 01:44:40.425428 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:40.425405    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-14de36-76d6cf56d6-s52rn"]
Sep 19 01:44:41.063654 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:41.063618    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65" path="/var/lib/kubelet/pods/c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65/volumes"
Sep 19 01:44:57.482836 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:57.482800    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"]
Sep 19 01:44:57.483273 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:57.483022    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr" podUID="65331bea-956d-4ab0-aff8-ca56a9f6b517" containerName="maas-api" containerID="cri-o://449bc2d3267f87bb94a8c467c77abfdf4bb4283a572efac9cb18e6eb7069c836" gracePeriod=30
Sep 19 01:44:57.746387 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:57.746330    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"
Sep 19 01:44:57.769962 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:57.769922    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65331bea-956d-4ab0-aff8-ca56a9f6b517-openshift-service-ca\") pod \"65331bea-956d-4ab0-aff8-ca56a9f6b517\" (UID: \"65331bea-956d-4ab0-aff8-ca56a9f6b517\") "
Sep 19 01:44:57.770117 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:57.769982    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nscnr\" (UniqueName: \"kubernetes.io/projected/65331bea-956d-4ab0-aff8-ca56a9f6b517-kube-api-access-nscnr\") pod \"65331bea-956d-4ab0-aff8-ca56a9f6b517\" (UID: \"65331bea-956d-4ab0-aff8-ca56a9f6b517\") "
Sep 19 01:44:57.770117 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:57.770019    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65331bea-956d-4ab0-aff8-ca56a9f6b517-metrics-certs\") pod \"65331bea-956d-4ab0-aff8-ca56a9f6b517\" (UID: \"65331bea-956d-4ab0-aff8-ca56a9f6b517\") "
Sep 19 01:44:57.770117 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:57.770077    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65331bea-956d-4ab0-aff8-ca56a9f6b517-maas-api-tls\") pod \"65331bea-956d-4ab0-aff8-ca56a9f6b517\" (UID: \"65331bea-956d-4ab0-aff8-ca56a9f6b517\") "
Sep 19 01:44:57.770344 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:57.770317    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65331bea-956d-4ab0-aff8-ca56a9f6b517-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "65331bea-956d-4ab0-aff8-ca56a9f6b517" (UID: "65331bea-956d-4ab0-aff8-ca56a9f6b517"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:44:57.770784 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:57.770402    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65331bea-956d-4ab0-aff8-ca56a9f6b517-openshift-service-ca\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:44:57.773033 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:57.772984    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65331bea-956d-4ab0-aff8-ca56a9f6b517-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "65331bea-956d-4ab0-aff8-ca56a9f6b517" (UID: "65331bea-956d-4ab0-aff8-ca56a9f6b517"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:44:57.773761 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:57.773709    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65331bea-956d-4ab0-aff8-ca56a9f6b517-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "65331bea-956d-4ab0-aff8-ca56a9f6b517" (UID: "65331bea-956d-4ab0-aff8-ca56a9f6b517"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:44:57.773948 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:57.773860    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65331bea-956d-4ab0-aff8-ca56a9f6b517-kube-api-access-nscnr" (OuterVolumeSpecName: "kube-api-access-nscnr") pod "65331bea-956d-4ab0-aff8-ca56a9f6b517" (UID: "65331bea-956d-4ab0-aff8-ca56a9f6b517"). InnerVolumeSpecName "kube-api-access-nscnr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:44:57.871652 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:57.871618    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nscnr\" (UniqueName: \"kubernetes.io/projected/65331bea-956d-4ab0-aff8-ca56a9f6b517-kube-api-access-nscnr\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:44:57.871652 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:57.871647    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65331bea-956d-4ab0-aff8-ca56a9f6b517-metrics-certs\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:44:57.871652 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:57.871658    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65331bea-956d-4ab0-aff8-ca56a9f6b517-maas-api-tls\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:44:58.514347 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:58.514314    2580 generic.go:356] "Generic (PLEG): container finished" podID="65331bea-956d-4ab0-aff8-ca56a9f6b517" containerID="449bc2d3267f87bb94a8c467c77abfdf4bb4283a572efac9cb18e6eb7069c836" exitCode=0
Sep 19 01:44:58.514924 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:58.514357    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"
Sep 19 01:44:58.514924 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:58.514398    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr" event={"ID":"65331bea-956d-4ab0-aff8-ca56a9f6b517","Type":"ContainerDied","Data":"449bc2d3267f87bb94a8c467c77abfdf4bb4283a572efac9cb18e6eb7069c836"}
Sep 19 01:44:58.514924 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:58.514460    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr" event={"ID":"65331bea-956d-4ab0-aff8-ca56a9f6b517","Type":"ContainerDied","Data":"73b31d252aea9b99fa1cded62daf6f0118a520c1a45aca08517472cabb0468f2"}
Sep 19 01:44:58.514924 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:58.514522    2580 scope.go:117] "RemoveContainer" containerID="449bc2d3267f87bb94a8c467c77abfdf4bb4283a572efac9cb18e6eb7069c836"
Sep 19 01:44:58.524719 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:58.524699    2580 scope.go:117] "RemoveContainer" containerID="449bc2d3267f87bb94a8c467c77abfdf4bb4283a572efac9cb18e6eb7069c836"
Sep 19 01:44:58.525030 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:44:58.525012    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"449bc2d3267f87bb94a8c467c77abfdf4bb4283a572efac9cb18e6eb7069c836\": container with ID starting with 449bc2d3267f87bb94a8c467c77abfdf4bb4283a572efac9cb18e6eb7069c836 not found: ID does not exist" containerID="449bc2d3267f87bb94a8c467c77abfdf4bb4283a572efac9cb18e6eb7069c836"
Sep 19 01:44:58.525096 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:58.525036    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"449bc2d3267f87bb94a8c467c77abfdf4bb4283a572efac9cb18e6eb7069c836"} err="failed to get container status \"449bc2d3267f87bb94a8c467c77abfdf4bb4283a572efac9cb18e6eb7069c836\": rpc error: code = NotFound desc = could not find container \"449bc2d3267f87bb94a8c467c77abfdf4bb4283a572efac9cb18e6eb7069c836\": container with ID starting with 449bc2d3267f87bb94a8c467c77abfdf4bb4283a572efac9cb18e6eb7069c836 not found: ID does not exist"
Sep 19 01:44:58.536791 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:58.536770    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"]
Sep 19 01:44:58.543270 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:58.543237    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-751f6a-6689d46d7c-blpdr"]
Sep 19 01:44:59.065052 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:44:59.065020    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65331bea-956d-4ab0-aff8-ca56a9f6b517" path="/var/lib/kubelet/pods/65331bea-956d-4ab0-aff8-ca56a9f6b517/volumes"
Sep 19 01:45:16.815929 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:16.815847    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"]
Sep 19 01:45:16.816397 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:16.816084    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq" podUID="f39221aa-43db-4aba-871e-0d285e983c73" containerName="maas-api" containerID="cri-o://717633fe1deb0c8b420c8298057f896aa6992b11edc9928817e091f574b447f1" gracePeriod=30
Sep 19 01:45:16.883369 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:16.883338    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh"]
Sep 19 01:45:16.883599 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:16.883572    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh" podUID="a3adc67a-5992-497a-995b-7e855970367b" containerName="payload-pre-processing" containerID="cri-o://ccc6c234855316c34b0e8cf01e10c551752379ac1d032f8ce6af6a190a1a82dd" gracePeriod=30
Sep 19 01:45:17.144074 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.144037    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"
Sep 19 01:45:17.181905 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.181882    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh"
Sep 19 01:45:17.245103 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.245078    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f39221aa-43db-4aba-871e-0d285e983c73-maas-api-tls\") pod \"f39221aa-43db-4aba-871e-0d285e983c73\" (UID: \"f39221aa-43db-4aba-871e-0d285e983c73\") "
Sep 19 01:45:17.245281 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.245134    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3adc67a-5992-497a-995b-7e855970367b-plugins-config-volume\") pod \"a3adc67a-5992-497a-995b-7e855970367b\" (UID: \"a3adc67a-5992-497a-995b-7e855970367b\") "
Sep 19 01:45:17.245281 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.245165    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f39221aa-43db-4aba-871e-0d285e983c73-openshift-service-ca\") pod \"f39221aa-43db-4aba-871e-0d285e983c73\" (UID: \"f39221aa-43db-4aba-871e-0d285e983c73\") "
Sep 19 01:45:17.245281 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.245190    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zk58c\" (UniqueName: \"kubernetes.io/projected/a3adc67a-5992-497a-995b-7e855970367b-kube-api-access-zk58c\") pod \"a3adc67a-5992-497a-995b-7e855970367b\" (UID: \"a3adc67a-5992-497a-995b-7e855970367b\") "
Sep 19 01:45:17.245281 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.245226    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmcln\" (UniqueName: \"kubernetes.io/projected/f39221aa-43db-4aba-871e-0d285e983c73-kube-api-access-pmcln\") pod \"f39221aa-43db-4aba-871e-0d285e983c73\" (UID: \"f39221aa-43db-4aba-871e-0d285e983c73\") "
Sep 19 01:45:17.245281 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.245249    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f39221aa-43db-4aba-871e-0d285e983c73-metrics-certs\") pod \"f39221aa-43db-4aba-871e-0d285e983c73\" (UID: \"f39221aa-43db-4aba-871e-0d285e983c73\") "
Sep 19 01:45:17.245617 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.245590    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3adc67a-5992-497a-995b-7e855970367b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a3adc67a-5992-497a-995b-7e855970367b" (UID: "a3adc67a-5992-497a-995b-7e855970367b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:45:17.246047 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.246016    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f39221aa-43db-4aba-871e-0d285e983c73-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f39221aa-43db-4aba-871e-0d285e983c73" (UID: "f39221aa-43db-4aba-871e-0d285e983c73"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:45:17.247433 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.247403    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f39221aa-43db-4aba-871e-0d285e983c73-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f39221aa-43db-4aba-871e-0d285e983c73" (UID: "f39221aa-43db-4aba-871e-0d285e983c73"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:45:17.247575 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.247433    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f39221aa-43db-4aba-871e-0d285e983c73-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f39221aa-43db-4aba-871e-0d285e983c73" (UID: "f39221aa-43db-4aba-871e-0d285e983c73"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:45:17.247812 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.247793    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f39221aa-43db-4aba-871e-0d285e983c73-kube-api-access-pmcln" (OuterVolumeSpecName: "kube-api-access-pmcln") pod "f39221aa-43db-4aba-871e-0d285e983c73" (UID: "f39221aa-43db-4aba-871e-0d285e983c73"). InnerVolumeSpecName "kube-api-access-pmcln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:45:17.247858 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.247819    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3adc67a-5992-497a-995b-7e855970367b-kube-api-access-zk58c" (OuterVolumeSpecName: "kube-api-access-zk58c") pod "a3adc67a-5992-497a-995b-7e855970367b" (UID: "a3adc67a-5992-497a-995b-7e855970367b"). InnerVolumeSpecName "kube-api-access-zk58c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:45:17.346755 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.346718    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f39221aa-43db-4aba-871e-0d285e983c73-maas-api-tls\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:45:17.346755 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.346757    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3adc67a-5992-497a-995b-7e855970367b-plugins-config-volume\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:45:17.346956 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.346772    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f39221aa-43db-4aba-871e-0d285e983c73-openshift-service-ca\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:45:17.346956 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.346787    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zk58c\" (UniqueName: \"kubernetes.io/projected/a3adc67a-5992-497a-995b-7e855970367b-kube-api-access-zk58c\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:45:17.346956 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.346801    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmcln\" (UniqueName: \"kubernetes.io/projected/f39221aa-43db-4aba-871e-0d285e983c73-kube-api-access-pmcln\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:45:17.346956 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.346815    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f39221aa-43db-4aba-871e-0d285e983c73-metrics-certs\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:45:17.640017 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.639979    2580 generic.go:356] "Generic (PLEG): container finished" podID="a3adc67a-5992-497a-995b-7e855970367b" containerID="ccc6c234855316c34b0e8cf01e10c551752379ac1d032f8ce6af6a190a1a82dd" exitCode=0
Sep 19 01:45:17.640228 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.640023    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh"
Sep 19 01:45:17.640228 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.640058    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh" event={"ID":"a3adc67a-5992-497a-995b-7e855970367b","Type":"ContainerDied","Data":"ccc6c234855316c34b0e8cf01e10c551752379ac1d032f8ce6af6a190a1a82dd"}
Sep 19 01:45:17.640228 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.640114    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh" event={"ID":"a3adc67a-5992-497a-995b-7e855970367b","Type":"ContainerDied","Data":"fa7ac07cec66793340ce97baa338da7129030b5223c0d72c5a83efb7e1e72f6a"}
Sep 19 01:45:17.640228 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.640144    2580 scope.go:117] "RemoveContainer" containerID="ccc6c234855316c34b0e8cf01e10c551752379ac1d032f8ce6af6a190a1a82dd"
Sep 19 01:45:17.641990 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.641962    2580 generic.go:356] "Generic (PLEG): container finished" podID="f39221aa-43db-4aba-871e-0d285e983c73" containerID="717633fe1deb0c8b420c8298057f896aa6992b11edc9928817e091f574b447f1" exitCode=0
Sep 19 01:45:17.642103 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.642000    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq" event={"ID":"f39221aa-43db-4aba-871e-0d285e983c73","Type":"ContainerDied","Data":"717633fe1deb0c8b420c8298057f896aa6992b11edc9928817e091f574b447f1"}
Sep 19 01:45:17.642103 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.642055    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq" event={"ID":"f39221aa-43db-4aba-871e-0d285e983c73","Type":"ContainerDied","Data":"29958c09a3dd20224c54e514a7ad6ac0236d74af7b48ba91c328445418f15ca0"}
Sep 19 01:45:17.642103 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.642007    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"
Sep 19 01:45:17.651303 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.651288    2580 scope.go:117] "RemoveContainer" containerID="ccc6c234855316c34b0e8cf01e10c551752379ac1d032f8ce6af6a190a1a82dd"
Sep 19 01:45:17.651664 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:45:17.651644    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccc6c234855316c34b0e8cf01e10c551752379ac1d032f8ce6af6a190a1a82dd\": container with ID starting with ccc6c234855316c34b0e8cf01e10c551752379ac1d032f8ce6af6a190a1a82dd not found: ID does not exist" containerID="ccc6c234855316c34b0e8cf01e10c551752379ac1d032f8ce6af6a190a1a82dd"
Sep 19 01:45:17.651771 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.651671    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccc6c234855316c34b0e8cf01e10c551752379ac1d032f8ce6af6a190a1a82dd"} err="failed to get container status \"ccc6c234855316c34b0e8cf01e10c551752379ac1d032f8ce6af6a190a1a82dd\": rpc error: code = NotFound desc = could not find container \"ccc6c234855316c34b0e8cf01e10c551752379ac1d032f8ce6af6a190a1a82dd\": container with ID starting with ccc6c234855316c34b0e8cf01e10c551752379ac1d032f8ce6af6a190a1a82dd not found: ID does not exist"
Sep 19 01:45:17.651771 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.651688    2580 scope.go:117] "RemoveContainer" containerID="717633fe1deb0c8b420c8298057f896aa6992b11edc9928817e091f574b447f1"
Sep 19 01:45:17.660315 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.660299    2580 scope.go:117] "RemoveContainer" containerID="717633fe1deb0c8b420c8298057f896aa6992b11edc9928817e091f574b447f1"
Sep 19 01:45:17.660618 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:45:17.660598    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"717633fe1deb0c8b420c8298057f896aa6992b11edc9928817e091f574b447f1\": container with ID starting with 717633fe1deb0c8b420c8298057f896aa6992b11edc9928817e091f574b447f1 not found: ID does not exist" containerID="717633fe1deb0c8b420c8298057f896aa6992b11edc9928817e091f574b447f1"
Sep 19 01:45:17.660689 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.660624    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"717633fe1deb0c8b420c8298057f896aa6992b11edc9928817e091f574b447f1"} err="failed to get container status \"717633fe1deb0c8b420c8298057f896aa6992b11edc9928817e091f574b447f1\": rpc error: code = NotFound desc = could not find container \"717633fe1deb0c8b420c8298057f896aa6992b11edc9928817e091f574b447f1\": container with ID starting with 717633fe1deb0c8b420c8298057f896aa6992b11edc9928817e091f574b447f1 not found: ID does not exist"
Sep 19 01:45:17.663992 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.663968    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh"]
Sep 19 01:45:17.670160 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.670138    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bebef4-f96b8f658-265wh"]
Sep 19 01:45:17.685613 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.685585    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"]
Sep 19 01:45:17.688289 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:17.688268    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bebef4-689958f49d-nx8tq"]
Sep 19 01:45:19.064310 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:19.064274    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3adc67a-5992-497a-995b-7e855970367b" path="/var/lib/kubelet/pods/a3adc67a-5992-497a-995b-7e855970367b/volumes"
Sep 19 01:45:19.064723 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:19.064708    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f39221aa-43db-4aba-871e-0d285e983c73" path="/var/lib/kubelet/pods/f39221aa-43db-4aba-871e-0d285e983c73/volumes"
Sep 19 01:45:27.848649 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:27.848548    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5bc8cb64b6-9v4gl"]
Sep 19 01:45:27.849080 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:27.848797    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5bc8cb64b6-9v4gl" podUID="ed9df944-2204-4102-9b29-9bfb381fdab7" containerName="manager" containerID="cri-o://8754becaa61e8e3de2c9117dbf0696340809e559772e6b47c39433d697f9d09f" gracePeriod=30
Sep 19 01:45:28.195416 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:28.195392    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bc8cb64b6-9v4gl"
Sep 19 01:45:28.343469 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:28.343436    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlvnp\" (UniqueName: \"kubernetes.io/projected/ed9df944-2204-4102-9b29-9bfb381fdab7-kube-api-access-hlvnp\") pod \"ed9df944-2204-4102-9b29-9bfb381fdab7\" (UID: \"ed9df944-2204-4102-9b29-9bfb381fdab7\") "
Sep 19 01:45:28.343676 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:28.343484    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed9df944-2204-4102-9b29-9bfb381fdab7-webhook-cert\") pod \"ed9df944-2204-4102-9b29-9bfb381fdab7\" (UID: \"ed9df944-2204-4102-9b29-9bfb381fdab7\") "
Sep 19 01:45:28.343676 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:28.343544    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed9df944-2204-4102-9b29-9bfb381fdab7-metrics-certs\") pod \"ed9df944-2204-4102-9b29-9bfb381fdab7\" (UID: \"ed9df944-2204-4102-9b29-9bfb381fdab7\") "
Sep 19 01:45:28.345766 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:28.345738    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed9df944-2204-4102-9b29-9bfb381fdab7-kube-api-access-hlvnp" (OuterVolumeSpecName: "kube-api-access-hlvnp") pod "ed9df944-2204-4102-9b29-9bfb381fdab7" (UID: "ed9df944-2204-4102-9b29-9bfb381fdab7"). InnerVolumeSpecName "kube-api-access-hlvnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:45:28.345914 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:28.345764    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed9df944-2204-4102-9b29-9bfb381fdab7-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ed9df944-2204-4102-9b29-9bfb381fdab7" (UID: "ed9df944-2204-4102-9b29-9bfb381fdab7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:45:28.345914 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:28.345736    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed9df944-2204-4102-9b29-9bfb381fdab7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ed9df944-2204-4102-9b29-9bfb381fdab7" (UID: "ed9df944-2204-4102-9b29-9bfb381fdab7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:45:28.444953 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:28.444879    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlvnp\" (UniqueName: \"kubernetes.io/projected/ed9df944-2204-4102-9b29-9bfb381fdab7-kube-api-access-hlvnp\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:45:28.444953 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:28.444909    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed9df944-2204-4102-9b29-9bfb381fdab7-webhook-cert\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:45:28.444953 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:28.444919    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed9df944-2204-4102-9b29-9bfb381fdab7-metrics-certs\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:45:28.715770 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:28.715690    2580 generic.go:356] "Generic (PLEG): container finished" podID="ed9df944-2204-4102-9b29-9bfb381fdab7" containerID="8754becaa61e8e3de2c9117dbf0696340809e559772e6b47c39433d697f9d09f" exitCode=0
Sep 19 01:45:28.715770 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:28.715734    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bc8cb64b6-9v4gl"
Sep 19 01:45:28.715770 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:28.715767    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bc8cb64b6-9v4gl" event={"ID":"ed9df944-2204-4102-9b29-9bfb381fdab7","Type":"ContainerDied","Data":"8754becaa61e8e3de2c9117dbf0696340809e559772e6b47c39433d697f9d09f"}
Sep 19 01:45:28.716018 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:28.715807    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bc8cb64b6-9v4gl" event={"ID":"ed9df944-2204-4102-9b29-9bfb381fdab7","Type":"ContainerDied","Data":"d63487059bb7c5a93a4e53bb4a22421046a5d072456cfdda7b6ffbcd52b09dc2"}
Sep 19 01:45:28.716018 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:28.715828    2580 scope.go:117] "RemoveContainer" containerID="8754becaa61e8e3de2c9117dbf0696340809e559772e6b47c39433d697f9d09f"
Sep 19 01:45:28.726024 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:28.726009    2580 scope.go:117] "RemoveContainer" containerID="8754becaa61e8e3de2c9117dbf0696340809e559772e6b47c39433d697f9d09f"
Sep 19 01:45:28.726275 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:45:28.726257    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8754becaa61e8e3de2c9117dbf0696340809e559772e6b47c39433d697f9d09f\": container with ID starting with 8754becaa61e8e3de2c9117dbf0696340809e559772e6b47c39433d697f9d09f not found: ID does not exist" containerID="8754becaa61e8e3de2c9117dbf0696340809e559772e6b47c39433d697f9d09f"
Sep 19 01:45:28.726331 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:28.726282    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8754becaa61e8e3de2c9117dbf0696340809e559772e6b47c39433d697f9d09f"} err="failed to get container status \"8754becaa61e8e3de2c9117dbf0696340809e559772e6b47c39433d697f9d09f\": rpc error: code = NotFound desc = could not find container \"8754becaa61e8e3de2c9117dbf0696340809e559772e6b47c39433d697f9d09f\": container with ID starting with 8754becaa61e8e3de2c9117dbf0696340809e559772e6b47c39433d697f9d09f not found: ID does not exist"
Sep 19 01:45:28.744666 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:28.744646    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5bc8cb64b6-9v4gl"]
Sep 19 01:45:28.750703 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:28.750679    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5bc8cb64b6-9v4gl"]
Sep 19 01:45:29.063629 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:45:29.063597    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed9df944-2204-4102-9b29-9bfb381fdab7" path="/var/lib/kubelet/pods/ed9df944-2204-4102-9b29-9bfb381fdab7/volumes"
Sep 19 01:46:29.640929 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.640888    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5bc8cb64b6-n5rc8"]
Sep 19 01:46:29.641529 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.641484    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65" containerName="maas-api"
Sep 19 01:46:29.641529 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.641523    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65" containerName="maas-api"
Sep 19 01:46:29.641684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.641562    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4858af12-7b70-427b-b689-6a6c87f992fe" containerName="maas-api"
Sep 19 01:46:29.641684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.641573    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4858af12-7b70-427b-b689-6a6c87f992fe" containerName="maas-api"
Sep 19 01:46:29.641684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.641603    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed9df944-2204-4102-9b29-9bfb381fdab7" containerName="manager"
Sep 19 01:46:29.641684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.641612    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed9df944-2204-4102-9b29-9bfb381fdab7" containerName="manager"
Sep 19 01:46:29.641684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.641623    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65331bea-956d-4ab0-aff8-ca56a9f6b517" containerName="maas-api"
Sep 19 01:46:29.641684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.641632    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="65331bea-956d-4ab0-aff8-ca56a9f6b517" containerName="maas-api"
Sep 19 01:46:29.641684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.641642    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f39221aa-43db-4aba-871e-0d285e983c73" containerName="maas-api"
Sep 19 01:46:29.641684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.641651    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f39221aa-43db-4aba-871e-0d285e983c73" containerName="maas-api"
Sep 19 01:46:29.641684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.641663    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31c799a2-1a45-4af6-b751-993ca034a4bf" containerName="maas-api"
Sep 19 01:46:29.641684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.641672    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="31c799a2-1a45-4af6-b751-993ca034a4bf" containerName="maas-api"
Sep 19 01:46:29.641684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.641681    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3adc67a-5992-497a-995b-7e855970367b" containerName="payload-pre-processing"
Sep 19 01:46:29.642219 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.641690    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3adc67a-5992-497a-995b-7e855970367b" containerName="payload-pre-processing"
Sep 19 01:46:29.642219 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.641800    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f39221aa-43db-4aba-871e-0d285e983c73" containerName="maas-api"
Sep 19 01:46:29.642219 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.641815    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4858af12-7b70-427b-b689-6a6c87f992fe" containerName="maas-api"
Sep 19 01:46:29.642219 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.641826    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9bc2cdd-75b9-44d0-90c3-a7ff6b4c6d65" containerName="maas-api"
Sep 19 01:46:29.642219 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.641835    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="65331bea-956d-4ab0-aff8-ca56a9f6b517" containerName="maas-api"
Sep 19 01:46:29.642219 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.641843    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="31c799a2-1a45-4af6-b751-993ca034a4bf" containerName="maas-api"
Sep 19 01:46:29.642219 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.641851    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3adc67a-5992-497a-995b-7e855970367b" containerName="payload-pre-processing"
Sep 19 01:46:29.642219 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.641862    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed9df944-2204-4102-9b29-9bfb381fdab7" containerName="manager"
Sep 19 01:46:29.645570 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.645549    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bc8cb64b6-n5rc8"
Sep 19 01:46:29.648757 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.648736    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 19 01:46:29.648877 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.648856    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 19 01:46:29.649454 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.649441    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 19 01:46:29.649738 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.649723    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-v6j6j\""
Sep 19 01:46:29.661966 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.661942    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bc8cb64b6-n5rc8"]
Sep 19 01:46:29.810769 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.810733    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c328524e-cbd1-478b-904f-27d39638e7b9-webhook-cert\") pod \"maas-controller-5bc8cb64b6-n5rc8\" (UID: \"c328524e-cbd1-478b-904f-27d39638e7b9\") " pod="opendatahub/maas-controller-5bc8cb64b6-n5rc8"
Sep 19 01:46:29.810769 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.810768    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bjqs\" (UniqueName: \"kubernetes.io/projected/c328524e-cbd1-478b-904f-27d39638e7b9-kube-api-access-7bjqs\") pod \"maas-controller-5bc8cb64b6-n5rc8\" (UID: \"c328524e-cbd1-478b-904f-27d39638e7b9\") " pod="opendatahub/maas-controller-5bc8cb64b6-n5rc8"
Sep 19 01:46:29.811032 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.810890    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c328524e-cbd1-478b-904f-27d39638e7b9-metrics-certs\") pod \"maas-controller-5bc8cb64b6-n5rc8\" (UID: \"c328524e-cbd1-478b-904f-27d39638e7b9\") " pod="opendatahub/maas-controller-5bc8cb64b6-n5rc8"
Sep 19 01:46:29.912107 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.912001    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c328524e-cbd1-478b-904f-27d39638e7b9-webhook-cert\") pod \"maas-controller-5bc8cb64b6-n5rc8\" (UID: \"c328524e-cbd1-478b-904f-27d39638e7b9\") " pod="opendatahub/maas-controller-5bc8cb64b6-n5rc8"
Sep 19 01:46:29.912107 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.912051    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bjqs\" (UniqueName: \"kubernetes.io/projected/c328524e-cbd1-478b-904f-27d39638e7b9-kube-api-access-7bjqs\") pod \"maas-controller-5bc8cb64b6-n5rc8\" (UID: \"c328524e-cbd1-478b-904f-27d39638e7b9\") " pod="opendatahub/maas-controller-5bc8cb64b6-n5rc8"
Sep 19 01:46:29.912357 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.912121    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c328524e-cbd1-478b-904f-27d39638e7b9-metrics-certs\") pod \"maas-controller-5bc8cb64b6-n5rc8\" (UID: \"c328524e-cbd1-478b-904f-27d39638e7b9\") " pod="opendatahub/maas-controller-5bc8cb64b6-n5rc8"
Sep 19 01:46:29.914481 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.914456    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c328524e-cbd1-478b-904f-27d39638e7b9-metrics-certs\") pod \"maas-controller-5bc8cb64b6-n5rc8\" (UID: \"c328524e-cbd1-478b-904f-27d39638e7b9\") " pod="opendatahub/maas-controller-5bc8cb64b6-n5rc8"
Sep 19 01:46:29.914600 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.914546    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c328524e-cbd1-478b-904f-27d39638e7b9-webhook-cert\") pod \"maas-controller-5bc8cb64b6-n5rc8\" (UID: \"c328524e-cbd1-478b-904f-27d39638e7b9\") " pod="opendatahub/maas-controller-5bc8cb64b6-n5rc8"
Sep 19 01:46:29.926990 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.926970    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bjqs\" (UniqueName: \"kubernetes.io/projected/c328524e-cbd1-478b-904f-27d39638e7b9-kube-api-access-7bjqs\") pod \"maas-controller-5bc8cb64b6-n5rc8\" (UID: \"c328524e-cbd1-478b-904f-27d39638e7b9\") " pod="opendatahub/maas-controller-5bc8cb64b6-n5rc8"
Sep 19 01:46:29.956221 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:29.956199    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bc8cb64b6-n5rc8"
Sep 19 01:46:30.251934 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:30.251909    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bc8cb64b6-n5rc8"]
Sep 19 01:46:30.253551 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:46:30.253522    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc328524e_cbd1_478b_904f_27d39638e7b9.slice/crio-f2d648e8c1bc75d9a7191caa3c34eca89b954af6f0ea94734a94d77dbc3c068d WatchSource:0}: Error finding container f2d648e8c1bc75d9a7191caa3c34eca89b954af6f0ea94734a94d77dbc3c068d: Status 404 returned error can't find the container with id f2d648e8c1bc75d9a7191caa3c34eca89b954af6f0ea94734a94d77dbc3c068d
Sep 19 01:46:31.083893 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:31.083858    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bc8cb64b6-n5rc8" event={"ID":"c328524e-cbd1-478b-904f-27d39638e7b9","Type":"ContainerStarted","Data":"7d31fe9041cf791bdc0f8ba1ce9c0b6f3b7412c34cc88b8318fdd0fd7452cc78"}
Sep 19 01:46:31.084364 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:31.083916    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bc8cb64b6-n5rc8" event={"ID":"c328524e-cbd1-478b-904f-27d39638e7b9","Type":"ContainerStarted","Data":"f2d648e8c1bc75d9a7191caa3c34eca89b954af6f0ea94734a94d77dbc3c068d"}
Sep 19 01:46:31.084364 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:31.084038    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5bc8cb64b6-n5rc8"
Sep 19 01:46:31.122317 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:31.122236    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5bc8cb64b6-n5rc8" podStartSLOduration=1.5399491379999999 podStartE2EDuration="2.122222057s" totalImagesPullingTime="582.272919ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:46:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:46:31.120403021 +0000 UTC m=+2046.027265069" watchObservedRunningTime="2026-09-19 01:46:31.122222057 +0000 UTC m=+2046.029084104"
Sep 19 01:46:42.092229 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:42.092149    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5bc8cb64b6-n5rc8"
Sep 19 01:46:53.596552 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:53.596511    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5bc8cb64b6-n5rc8"]
Sep 19 01:46:53.597035 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:53.596768    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5bc8cb64b6-n5rc8" podUID="c328524e-cbd1-478b-904f-27d39638e7b9" containerName="manager" containerID="cri-o://7d31fe9041cf791bdc0f8ba1ce9c0b6f3b7412c34cc88b8318fdd0fd7452cc78" gracePeriod=30
Sep 19 01:46:53.845348 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:53.845326    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bc8cb64b6-n5rc8"
Sep 19 01:46:53.919626 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:53.919555    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c328524e-cbd1-478b-904f-27d39638e7b9-metrics-certs\") pod \"c328524e-cbd1-478b-904f-27d39638e7b9\" (UID: \"c328524e-cbd1-478b-904f-27d39638e7b9\") "
Sep 19 01:46:53.919626 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:53.919602    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c328524e-cbd1-478b-904f-27d39638e7b9-webhook-cert\") pod \"c328524e-cbd1-478b-904f-27d39638e7b9\" (UID: \"c328524e-cbd1-478b-904f-27d39638e7b9\") "
Sep 19 01:46:53.919802 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:53.919662    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bjqs\" (UniqueName: \"kubernetes.io/projected/c328524e-cbd1-478b-904f-27d39638e7b9-kube-api-access-7bjqs\") pod \"c328524e-cbd1-478b-904f-27d39638e7b9\" (UID: \"c328524e-cbd1-478b-904f-27d39638e7b9\") "
Sep 19 01:46:53.921729 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:53.921702    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c328524e-cbd1-478b-904f-27d39638e7b9-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c328524e-cbd1-478b-904f-27d39638e7b9" (UID: "c328524e-cbd1-478b-904f-27d39638e7b9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:46:53.921729 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:53.921712    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c328524e-cbd1-478b-904f-27d39638e7b9-kube-api-access-7bjqs" (OuterVolumeSpecName: "kube-api-access-7bjqs") pod "c328524e-cbd1-478b-904f-27d39638e7b9" (UID: "c328524e-cbd1-478b-904f-27d39638e7b9"). InnerVolumeSpecName "kube-api-access-7bjqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:46:53.921729 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:53.921723    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c328524e-cbd1-478b-904f-27d39638e7b9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c328524e-cbd1-478b-904f-27d39638e7b9" (UID: "c328524e-cbd1-478b-904f-27d39638e7b9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:46:54.021182 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:54.021152    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c328524e-cbd1-478b-904f-27d39638e7b9-metrics-certs\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:46:54.021182 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:54.021177    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c328524e-cbd1-478b-904f-27d39638e7b9-webhook-cert\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:46:54.021182 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:54.021186    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bjqs\" (UniqueName: \"kubernetes.io/projected/c328524e-cbd1-478b-904f-27d39638e7b9-kube-api-access-7bjqs\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:46:54.215974 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:54.215889    2580 generic.go:356] "Generic (PLEG): container finished" podID="c328524e-cbd1-478b-904f-27d39638e7b9" containerID="7d31fe9041cf791bdc0f8ba1ce9c0b6f3b7412c34cc88b8318fdd0fd7452cc78" exitCode=0
Sep 19 01:46:54.215974 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:54.215931    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bc8cb64b6-n5rc8"
Sep 19 01:46:54.216185 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:54.215973    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bc8cb64b6-n5rc8" event={"ID":"c328524e-cbd1-478b-904f-27d39638e7b9","Type":"ContainerDied","Data":"7d31fe9041cf791bdc0f8ba1ce9c0b6f3b7412c34cc88b8318fdd0fd7452cc78"}
Sep 19 01:46:54.216185 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:54.216021    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bc8cb64b6-n5rc8" event={"ID":"c328524e-cbd1-478b-904f-27d39638e7b9","Type":"ContainerDied","Data":"f2d648e8c1bc75d9a7191caa3c34eca89b954af6f0ea94734a94d77dbc3c068d"}
Sep 19 01:46:54.216185 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:54.216045    2580 scope.go:117] "RemoveContainer" containerID="7d31fe9041cf791bdc0f8ba1ce9c0b6f3b7412c34cc88b8318fdd0fd7452cc78"
Sep 19 01:46:54.226355 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:54.226337    2580 scope.go:117] "RemoveContainer" containerID="7d31fe9041cf791bdc0f8ba1ce9c0b6f3b7412c34cc88b8318fdd0fd7452cc78"
Sep 19 01:46:54.226706 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:46:54.226684    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d31fe9041cf791bdc0f8ba1ce9c0b6f3b7412c34cc88b8318fdd0fd7452cc78\": container with ID starting with 7d31fe9041cf791bdc0f8ba1ce9c0b6f3b7412c34cc88b8318fdd0fd7452cc78 not found: ID does not exist" containerID="7d31fe9041cf791bdc0f8ba1ce9c0b6f3b7412c34cc88b8318fdd0fd7452cc78"
Sep 19 01:46:54.226799 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:54.226715    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d31fe9041cf791bdc0f8ba1ce9c0b6f3b7412c34cc88b8318fdd0fd7452cc78"} err="failed to get container status \"7d31fe9041cf791bdc0f8ba1ce9c0b6f3b7412c34cc88b8318fdd0fd7452cc78\": rpc error: code = NotFound desc = could not find container \"7d31fe9041cf791bdc0f8ba1ce9c0b6f3b7412c34cc88b8318fdd0fd7452cc78\": container with ID starting with 7d31fe9041cf791bdc0f8ba1ce9c0b6f3b7412c34cc88b8318fdd0fd7452cc78 not found: ID does not exist"
Sep 19 01:46:54.242201 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:54.242178    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5bc8cb64b6-n5rc8"]
Sep 19 01:46:54.247483 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:54.247463    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5bc8cb64b6-n5rc8"]
Sep 19 01:46:55.053376 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:55.053335    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5bc8cb64b6-8cqf4"]
Sep 19 01:46:55.054108 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:55.054083    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c328524e-cbd1-478b-904f-27d39638e7b9" containerName="manager"
Sep 19 01:46:55.054108 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:55.054106    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c328524e-cbd1-478b-904f-27d39638e7b9" containerName="manager"
Sep 19 01:46:55.054290 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:55.054202    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c328524e-cbd1-478b-904f-27d39638e7b9" containerName="manager"
Sep 19 01:46:55.058728 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:55.058702    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bc8cb64b6-8cqf4"
Sep 19 01:46:55.061604 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:55.061584    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 19 01:46:55.061719 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:55.061654    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 19 01:46:55.061790 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:55.061720    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-v6j6j\""
Sep 19 01:46:55.061790 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:55.061721    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 19 01:46:55.068069 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:55.068045    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c328524e-cbd1-478b-904f-27d39638e7b9" path="/var/lib/kubelet/pods/c328524e-cbd1-478b-904f-27d39638e7b9/volumes"
Sep 19 01:46:55.068442 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:55.068425    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bc8cb64b6-8cqf4"]
Sep 19 01:46:55.131737 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:55.131708    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/436955fc-9575-4cb2-bcdc-dec43a78bec6-metrics-certs\") pod \"maas-controller-5bc8cb64b6-8cqf4\" (UID: \"436955fc-9575-4cb2-bcdc-dec43a78bec6\") " pod="opendatahub/maas-controller-5bc8cb64b6-8cqf4"
Sep 19 01:46:55.131892 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:55.131847    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmwtb\" (UniqueName: \"kubernetes.io/projected/436955fc-9575-4cb2-bcdc-dec43a78bec6-kube-api-access-lmwtb\") pod \"maas-controller-5bc8cb64b6-8cqf4\" (UID: \"436955fc-9575-4cb2-bcdc-dec43a78bec6\") " pod="opendatahub/maas-controller-5bc8cb64b6-8cqf4"
Sep 19 01:46:55.131892 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:55.131875    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/436955fc-9575-4cb2-bcdc-dec43a78bec6-webhook-cert\") pod \"maas-controller-5bc8cb64b6-8cqf4\" (UID: \"436955fc-9575-4cb2-bcdc-dec43a78bec6\") " pod="opendatahub/maas-controller-5bc8cb64b6-8cqf4"
Sep 19 01:46:55.232597 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:55.232565    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmwtb\" (UniqueName: \"kubernetes.io/projected/436955fc-9575-4cb2-bcdc-dec43a78bec6-kube-api-access-lmwtb\") pod \"maas-controller-5bc8cb64b6-8cqf4\" (UID: \"436955fc-9575-4cb2-bcdc-dec43a78bec6\") " pod="opendatahub/maas-controller-5bc8cb64b6-8cqf4"
Sep 19 01:46:55.232597 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:55.232602    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/436955fc-9575-4cb2-bcdc-dec43a78bec6-webhook-cert\") pod \"maas-controller-5bc8cb64b6-8cqf4\" (UID: \"436955fc-9575-4cb2-bcdc-dec43a78bec6\") " pod="opendatahub/maas-controller-5bc8cb64b6-8cqf4"
Sep 19 01:46:55.232851 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:55.232675    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/436955fc-9575-4cb2-bcdc-dec43a78bec6-metrics-certs\") pod \"maas-controller-5bc8cb64b6-8cqf4\" (UID: \"436955fc-9575-4cb2-bcdc-dec43a78bec6\") " pod="opendatahub/maas-controller-5bc8cb64b6-8cqf4"
Sep 19 01:46:55.235289 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:55.235263    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/436955fc-9575-4cb2-bcdc-dec43a78bec6-webhook-cert\") pod \"maas-controller-5bc8cb64b6-8cqf4\" (UID: \"436955fc-9575-4cb2-bcdc-dec43a78bec6\") " pod="opendatahub/maas-controller-5bc8cb64b6-8cqf4"
Sep 19 01:46:55.235384 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:55.235293    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/436955fc-9575-4cb2-bcdc-dec43a78bec6-metrics-certs\") pod \"maas-controller-5bc8cb64b6-8cqf4\" (UID: \"436955fc-9575-4cb2-bcdc-dec43a78bec6\") " pod="opendatahub/maas-controller-5bc8cb64b6-8cqf4"
Sep 19 01:46:55.241635 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:55.241608    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmwtb\" (UniqueName: \"kubernetes.io/projected/436955fc-9575-4cb2-bcdc-dec43a78bec6-kube-api-access-lmwtb\") pod \"maas-controller-5bc8cb64b6-8cqf4\" (UID: \"436955fc-9575-4cb2-bcdc-dec43a78bec6\") " pod="opendatahub/maas-controller-5bc8cb64b6-8cqf4"
Sep 19 01:46:55.371360 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:55.371282    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bc8cb64b6-8cqf4"
Sep 19 01:46:55.673405 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:55.673379    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bc8cb64b6-8cqf4"]
Sep 19 01:46:55.675154 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:46:55.675123    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod436955fc_9575_4cb2_bcdc_dec43a78bec6.slice/crio-5a982f2f0937ca6e76278213a1f0d661ee2f2c15e2c80639b21ae2a7c2661e55 WatchSource:0}: Error finding container 5a982f2f0937ca6e76278213a1f0d661ee2f2c15e2c80639b21ae2a7c2661e55: Status 404 returned error can't find the container with id 5a982f2f0937ca6e76278213a1f0d661ee2f2c15e2c80639b21ae2a7c2661e55
Sep 19 01:46:56.234244 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:56.234205    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bc8cb64b6-8cqf4" event={"ID":"436955fc-9575-4cb2-bcdc-dec43a78bec6","Type":"ContainerStarted","Data":"58b923d9c76098e72858d0cb84f986ce321407bfc5851418600bc47708ba8608"}
Sep 19 01:46:56.234649 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:56.234272    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bc8cb64b6-8cqf4" event={"ID":"436955fc-9575-4cb2-bcdc-dec43a78bec6","Type":"ContainerStarted","Data":"5a982f2f0937ca6e76278213a1f0d661ee2f2c15e2c80639b21ae2a7c2661e55"}
Sep 19 01:46:56.234649 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:56.234305    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5bc8cb64b6-8cqf4"
Sep 19 01:46:56.254930 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:46:56.254881    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5bc8cb64b6-8cqf4" podStartSLOduration=0.857762276 podStartE2EDuration="1.254864451s" totalImagesPullingTime="397.102175ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:46:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:46:56.25435555 +0000 UTC m=+2071.161217599" watchObservedRunningTime="2026-09-19 01:46:56.254864451 +0000 UTC m=+2071.161726506"
Sep 19 01:47:07.242900 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:47:07.242869    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5bc8cb64b6-8cqf4"
Sep 19 01:47:27.303943 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:47:27.303917    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f6bdh_74c8e2f6-1b01-4bd1-b491-ad407a15a109/console-operator/1.log"
Sep 19 01:47:27.313560 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:47:27.313535    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f6bdh_74c8e2f6-1b01-4bd1-b491-ad407a15a109/console-operator/1.log"
Sep 19 01:48:25.862538 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:48:25.862445    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zzrps"]
Sep 19 01:48:25.863017 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:48:25.862705    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zzrps" podUID="5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb" containerName="manager" containerID="cri-o://8a6c4180c14ab21e029612ac58a40988a9e1cac429fbb0458bd13ea04148e250" gracePeriod=10
Sep 19 01:48:26.124297 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:48:26.124235    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zzrps"
Sep 19 01:48:26.222247 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:48:26.222215    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb-extensions-socket-volume\") pod \"5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb\" (UID: \"5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb\") "
Sep 19 01:48:26.222434 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:48:26.222283    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mq2tj\" (UniqueName: \"kubernetes.io/projected/5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb-kube-api-access-mq2tj\") pod \"5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb\" (UID: \"5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb\") "
Sep 19 01:48:26.222647 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:48:26.222624    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb" (UID: "5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:48:26.224187 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:48:26.224165    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb-kube-api-access-mq2tj" (OuterVolumeSpecName: "kube-api-access-mq2tj") pod "5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb" (UID: "5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb"). InnerVolumeSpecName "kube-api-access-mq2tj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:48:26.322998 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:48:26.322966    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mq2tj\" (UniqueName: \"kubernetes.io/projected/5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb-kube-api-access-mq2tj\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:48:26.322998 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:48:26.322994    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb-extensions-socket-volume\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:48:26.794476 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:48:26.794434    2580 generic.go:356] "Generic (PLEG): container finished" podID="5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb" containerID="8a6c4180c14ab21e029612ac58a40988a9e1cac429fbb0458bd13ea04148e250" exitCode=0
Sep 19 01:48:26.794670 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:48:26.794522    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zzrps"
Sep 19 01:48:26.794670 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:48:26.794525    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zzrps" event={"ID":"5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb","Type":"ContainerDied","Data":"8a6c4180c14ab21e029612ac58a40988a9e1cac429fbb0458bd13ea04148e250"}
Sep 19 01:48:26.794670 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:48:26.794613    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zzrps" event={"ID":"5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb","Type":"ContainerDied","Data":"6d8c6db307bc7700ec153fb585a8d7775f9cfeb261749fd81aa12818edbaf110"}
Sep 19 01:48:26.794670 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:48:26.794636    2580 scope.go:117] "RemoveContainer" containerID="8a6c4180c14ab21e029612ac58a40988a9e1cac429fbb0458bd13ea04148e250"
Sep 19 01:48:26.804877 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:48:26.804861    2580 scope.go:117] "RemoveContainer" containerID="8a6c4180c14ab21e029612ac58a40988a9e1cac429fbb0458bd13ea04148e250"
Sep 19 01:48:26.805144 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:48:26.805129    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a6c4180c14ab21e029612ac58a40988a9e1cac429fbb0458bd13ea04148e250\": container with ID starting with 8a6c4180c14ab21e029612ac58a40988a9e1cac429fbb0458bd13ea04148e250 not found: ID does not exist" containerID="8a6c4180c14ab21e029612ac58a40988a9e1cac429fbb0458bd13ea04148e250"
Sep 19 01:48:26.805188 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:48:26.805151    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a6c4180c14ab21e029612ac58a40988a9e1cac429fbb0458bd13ea04148e250"} err="failed to get container status \"8a6c4180c14ab21e029612ac58a40988a9e1cac429fbb0458bd13ea04148e250\": rpc error: code = NotFound desc = could not find container \"8a6c4180c14ab21e029612ac58a40988a9e1cac429fbb0458bd13ea04148e250\": container with ID starting with 8a6c4180c14ab21e029612ac58a40988a9e1cac429fbb0458bd13ea04148e250 not found: ID does not exist"
Sep 19 01:48:26.818803 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:48:26.818781    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zzrps"]
Sep 19 01:48:26.826190 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:48:26.826172    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zzrps"]
Sep 19 01:48:27.064930 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:48:27.064856    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb" path="/var/lib/kubelet/pods/5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb/volumes"
Sep 19 01:49:26.576108 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:49:26.576067    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cpr4n"]
Sep 19 01:49:26.576915 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:49:26.576699    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb" containerName="manager"
Sep 19 01:49:26.576915 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:49:26.576721    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb" containerName="manager"
Sep 19 01:49:26.576915 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:49:26.576870    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c9a9323-c3dd-45cc-b7e0-cb4b7d8dfdeb" containerName="manager"
Sep 19 01:49:26.580216 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:49:26.580193    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cpr4n"
Sep 19 01:49:26.582817 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:49:26.582794    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nfzf4\""
Sep 19 01:49:26.597483 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:49:26.597459    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cpr4n"]
Sep 19 01:49:26.675961 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:49:26.675933    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn7zr\" (UniqueName: \"kubernetes.io/projected/2ed2e6bc-763f-474e-8eb5-0e730b7ce45d-kube-api-access-nn7zr\") pod \"kuadrant-operator-controller-manager-5769f99d9f-cpr4n\" (UID: \"2ed2e6bc-763f-474e-8eb5-0e730b7ce45d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cpr4n"
Sep 19 01:49:26.676132 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:49:26.675992    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ed2e6bc-763f-474e-8eb5-0e730b7ce45d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-cpr4n\" (UID: \"2ed2e6bc-763f-474e-8eb5-0e730b7ce45d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cpr4n"
Sep 19 01:49:26.777422 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:49:26.777389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn7zr\" (UniqueName: \"kubernetes.io/projected/2ed2e6bc-763f-474e-8eb5-0e730b7ce45d-kube-api-access-nn7zr\") pod \"kuadrant-operator-controller-manager-5769f99d9f-cpr4n\" (UID: \"2ed2e6bc-763f-474e-8eb5-0e730b7ce45d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cpr4n"
Sep 19 01:49:26.777588 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:49:26.777462    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ed2e6bc-763f-474e-8eb5-0e730b7ce45d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-cpr4n\" (UID: \"2ed2e6bc-763f-474e-8eb5-0e730b7ce45d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cpr4n"
Sep 19 01:49:26.777844 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:49:26.777824    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ed2e6bc-763f-474e-8eb5-0e730b7ce45d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-cpr4n\" (UID: \"2ed2e6bc-763f-474e-8eb5-0e730b7ce45d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cpr4n"
Sep 19 01:49:26.788221 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:49:26.788185    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn7zr\" (UniqueName: \"kubernetes.io/projected/2ed2e6bc-763f-474e-8eb5-0e730b7ce45d-kube-api-access-nn7zr\") pod \"kuadrant-operator-controller-manager-5769f99d9f-cpr4n\" (UID: \"2ed2e6bc-763f-474e-8eb5-0e730b7ce45d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cpr4n"
Sep 19 01:49:26.890724 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:49:26.890637    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cpr4n"
Sep 19 01:49:27.192140 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:49:27.192103    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ed2e6bc_763f_474e_8eb5_0e730b7ce45d.slice/crio-02a96b3da6e6433c3c0b01edb5dcafd3fc415e1bd24fb09071d3a33a95659ae5 WatchSource:0}: Error finding container 02a96b3da6e6433c3c0b01edb5dcafd3fc415e1bd24fb09071d3a33a95659ae5: Status 404 returned error can't find the container with id 02a96b3da6e6433c3c0b01edb5dcafd3fc415e1bd24fb09071d3a33a95659ae5
Sep 19 01:49:27.193354 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:49:27.193327    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cpr4n"]
Sep 19 01:49:28.151198 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:49:28.151160    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cpr4n" event={"ID":"2ed2e6bc-763f-474e-8eb5-0e730b7ce45d","Type":"ContainerStarted","Data":"34b3714f2e365b26cde553a6487b86922a002c875e798ef52331f90e40ac1efb"}
Sep 19 01:49:28.151639 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:49:28.151233    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cpr4n"
Sep 19 01:49:28.151639 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:49:28.151255    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cpr4n" event={"ID":"2ed2e6bc-763f-474e-8eb5-0e730b7ce45d","Type":"ContainerStarted","Data":"02a96b3da6e6433c3c0b01edb5dcafd3fc415e1bd24fb09071d3a33a95659ae5"}
Sep 19 01:49:28.189558 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:49:28.189490    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cpr4n" podStartSLOduration=2.189474447 podStartE2EDuration="2.189474447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:49:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:49:28.187207317 +0000 UTC m=+2223.094069366" watchObservedRunningTime="2026-09-19 01:49:28.189474447 +0000 UTC m=+2223.096336495"
Sep 19 01:49:39.159357 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:49:39.159327    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-cpr4n"
Sep 19 01:50:21.089605 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.089572    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"]
Sep 19 01:50:21.093321 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.093303    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:21.096572 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.096544    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-z77ck\""
Sep 19 01:50:21.096702 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.096544    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-0d586241-kserve-self-signed-certs\""
Sep 19 01:50:21.096702 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.096546    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 19 01:50:21.096800 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.096592    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 19 01:50:21.105680 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.105658    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"]
Sep 19 01:50:21.173755 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.173721    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-model-cache\") pod \"e2e-same-id-b-0d586241-kserve-55f599595f-ll87k\" (UID: \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\") " pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:21.173927 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.173852    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z42rw\" (UniqueName: \"kubernetes.io/projected/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-kube-api-access-z42rw\") pod \"e2e-same-id-b-0d586241-kserve-55f599595f-ll87k\" (UID: \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\") " pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:21.173927 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.173878    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-tmp-dir\") pod \"e2e-same-id-b-0d586241-kserve-55f599595f-ll87k\" (UID: \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\") " pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:21.174071 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.173989    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-tls-certs\") pod \"e2e-same-id-b-0d586241-kserve-55f599595f-ll87k\" (UID: \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\") " pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:21.174071 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.174037    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-home\") pod \"e2e-same-id-b-0d586241-kserve-55f599595f-ll87k\" (UID: \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\") " pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:21.174175 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.174085    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-dshm\") pod \"e2e-same-id-b-0d586241-kserve-55f599595f-ll87k\" (UID: \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\") " pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:21.274988 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.274951    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-tls-certs\") pod \"e2e-same-id-b-0d586241-kserve-55f599595f-ll87k\" (UID: \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\") " pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:21.274988 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.274994    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-home\") pod \"e2e-same-id-b-0d586241-kserve-55f599595f-ll87k\" (UID: \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\") " pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:21.275206 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.275025    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-dshm\") pod \"e2e-same-id-b-0d586241-kserve-55f599595f-ll87k\" (UID: \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\") " pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:21.275206 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.275073    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-model-cache\") pod \"e2e-same-id-b-0d586241-kserve-55f599595f-ll87k\" (UID: \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\") " pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:21.275206 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.275157    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z42rw\" (UniqueName: \"kubernetes.io/projected/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-kube-api-access-z42rw\") pod \"e2e-same-id-b-0d586241-kserve-55f599595f-ll87k\" (UID: \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\") " pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:21.275206 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.275181    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-tmp-dir\") pod \"e2e-same-id-b-0d586241-kserve-55f599595f-ll87k\" (UID: \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\") " pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:21.275556 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.275524    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-home\") pod \"e2e-same-id-b-0d586241-kserve-55f599595f-ll87k\" (UID: \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\") " pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:21.275690 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.275586    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-model-cache\") pod \"e2e-same-id-b-0d586241-kserve-55f599595f-ll87k\" (UID: \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\") " pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:21.275690 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.275595    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-tmp-dir\") pod \"e2e-same-id-b-0d586241-kserve-55f599595f-ll87k\" (UID: \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\") " pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:21.277428 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.277403    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-dshm\") pod \"e2e-same-id-b-0d586241-kserve-55f599595f-ll87k\" (UID: \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\") " pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:21.277728 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.277707    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-tls-certs\") pod \"e2e-same-id-b-0d586241-kserve-55f599595f-ll87k\" (UID: \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\") " pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:21.283101 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.283081    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z42rw\" (UniqueName: \"kubernetes.io/projected/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-kube-api-access-z42rw\") pod \"e2e-same-id-b-0d586241-kserve-55f599595f-ll87k\" (UID: \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\") " pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:21.405161 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.405073    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:21.682337 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:21.682312    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"]
Sep 19 01:50:21.684181 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:50:21.684138    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd96d0a0_e208_47e5_b6b0_e66a0567ae87.slice/crio-e938589f022742379412fc17f58105940d9b8e365e69643e91ffc41c61743b2f WatchSource:0}: Error finding container e938589f022742379412fc17f58105940d9b8e365e69643e91ffc41c61743b2f: Status 404 returned error can't find the container with id e938589f022742379412fc17f58105940d9b8e365e69643e91ffc41c61743b2f
Sep 19 01:50:22.494977 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:22.494944    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k" event={"ID":"cd96d0a0-e208-47e5-b6b0-e66a0567ae87","Type":"ContainerStarted","Data":"2f5b8b513619555f9a0a76fc5c28674024cf907deee131f51c1f7024979fee2f"}
Sep 19 01:50:22.495345 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:22.494984    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k" event={"ID":"cd96d0a0-e208-47e5-b6b0-e66a0567ae87","Type":"ContainerStarted","Data":"e938589f022742379412fc17f58105940d9b8e365e69643e91ffc41c61743b2f"}
Sep 19 01:50:22.526970 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:22.526919    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k" podStartSLOduration=1.526898333 podStartE2EDuration="1.526898333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:50:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:50:22.523561427 +0000 UTC m=+2277.430423475" watchObservedRunningTime="2026-09-19 01:50:22.526898333 +0000 UTC m=+2277.433760381"
Sep 19 01:50:31.405591 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:31.405556    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:31.405591 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:31.405599    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:31.418135 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:31.418115    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:31.564375 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:31.564349    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:41.523237 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:41.523198    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"]
Sep 19 01:50:56.543591 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:56.543544    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k" podUID="cd96d0a0-e208-47e5-b6b0-e66a0567ae87" containerName="main" containerID="cri-o://2f5b8b513619555f9a0a76fc5c28674024cf907deee131f51c1f7024979fee2f" gracePeriod=45
Sep 19 01:50:56.704688 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:56.704650    2580 generic.go:356] "Generic (PLEG): container finished" podID="cd96d0a0-e208-47e5-b6b0-e66a0567ae87" containerID="2f5b8b513619555f9a0a76fc5c28674024cf907deee131f51c1f7024979fee2f" exitCode=0
Sep 19 01:50:56.704854 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:56.704716    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k" event={"ID":"cd96d0a0-e208-47e5-b6b0-e66a0567ae87","Type":"ContainerDied","Data":"2f5b8b513619555f9a0a76fc5c28674024cf907deee131f51c1f7024979fee2f"}
Sep 19 01:50:56.802455 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:56.802393    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:56.903756 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:56.903721    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-model-cache\") pod \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\" (UID: \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\") "
Sep 19 01:50:56.903954 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:56.903771    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-tmp-dir\") pod \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\" (UID: \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\") "
Sep 19 01:50:56.903954 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:56.903802    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-home\") pod \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\" (UID: \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\") "
Sep 19 01:50:56.903954 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:56.903855    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-tls-certs\") pod \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\" (UID: \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\") "
Sep 19 01:50:56.903954 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:56.903890    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-dshm\") pod \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\" (UID: \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\") "
Sep 19 01:50:56.904193 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:56.904035    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z42rw\" (UniqueName: \"kubernetes.io/projected/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-kube-api-access-z42rw\") pod \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\" (UID: \"cd96d0a0-e208-47e5-b6b0-e66a0567ae87\") "
Sep 19 01:50:56.904193 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:56.904077    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cd96d0a0-e208-47e5-b6b0-e66a0567ae87" (UID: "cd96d0a0-e208-47e5-b6b0-e66a0567ae87"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:50:56.904193 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:56.904120    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-model-cache" (OuterVolumeSpecName: "model-cache") pod "cd96d0a0-e208-47e5-b6b0-e66a0567ae87" (UID: "cd96d0a0-e208-47e5-b6b0-e66a0567ae87"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:50:56.904193 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:56.904150    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-home" (OuterVolumeSpecName: "home") pod "cd96d0a0-e208-47e5-b6b0-e66a0567ae87" (UID: "cd96d0a0-e208-47e5-b6b0-e66a0567ae87"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:50:56.904386 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:56.904369    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-model-cache\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:50:56.904441 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:56.904389    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-tmp-dir\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:50:56.904441 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:56.904397    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-home\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:50:56.906238 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:56.906212    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cd96d0a0-e208-47e5-b6b0-e66a0567ae87" (UID: "cd96d0a0-e208-47e5-b6b0-e66a0567ae87"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:50:56.906342 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:56.906261    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-dshm" (OuterVolumeSpecName: "dshm") pod "cd96d0a0-e208-47e5-b6b0-e66a0567ae87" (UID: "cd96d0a0-e208-47e5-b6b0-e66a0567ae87"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 01:50:56.906342 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:56.906319    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-kube-api-access-z42rw" (OuterVolumeSpecName: "kube-api-access-z42rw") pod "cd96d0a0-e208-47e5-b6b0-e66a0567ae87" (UID: "cd96d0a0-e208-47e5-b6b0-e66a0567ae87"). InnerVolumeSpecName "kube-api-access-z42rw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:50:57.005271 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:57.005234    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-dshm\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:50:57.005271 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:57.005262    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z42rw\" (UniqueName: \"kubernetes.io/projected/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-kube-api-access-z42rw\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:50:57.005271 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:57.005273    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd96d0a0-e208-47e5-b6b0-e66a0567ae87-tls-certs\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:50:57.711844 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:57.711816    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"
Sep 19 01:50:57.712282 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:57.711815    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k" event={"ID":"cd96d0a0-e208-47e5-b6b0-e66a0567ae87","Type":"ContainerDied","Data":"e938589f022742379412fc17f58105940d9b8e365e69643e91ffc41c61743b2f"}
Sep 19 01:50:57.712282 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:57.711941    2580 scope.go:117] "RemoveContainer" containerID="2f5b8b513619555f9a0a76fc5c28674024cf907deee131f51c1f7024979fee2f"
Sep 19 01:50:57.728738 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:57.728713    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"]
Sep 19 01:50:57.732229 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:57.732205    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-0d586241-kserve-55f599595f-ll87k"]
Sep 19 01:50:59.063976 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:50:59.063935    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd96d0a0-e208-47e5-b6b0-e66a0567ae87" path="/var/lib/kubelet/pods/cd96d0a0-e208-47e5-b6b0-e66a0567ae87/volumes"
Sep 19 01:51:00.612054 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:00.612017    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss"]
Sep 19 01:51:00.612452 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:00.612439    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd96d0a0-e208-47e5-b6b0-e66a0567ae87" containerName="main"
Sep 19 01:51:00.612537 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:00.612454    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd96d0a0-e208-47e5-b6b0-e66a0567ae87" containerName="main"
Sep 19 01:51:00.612579 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:00.612565    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd96d0a0-e208-47e5-b6b0-e66a0567ae87" containerName="main"
Sep 19 01:51:00.617275 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:00.617255    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss"
Sep 19 01:51:00.620552 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:00.620515    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 19 01:51:00.620684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:00.620524    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 19 01:51:00.620684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:00.620566    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8d4ph\""
Sep 19 01:51:00.632298 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:00.632265    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss"]
Sep 19 01:51:00.740190 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:00.740155    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d065f53f-cb94-43a3-9625-dd5bac8bc2f9-metrics-certs\") pod \"maas-api-5598d8c9fb-bbkss\" (UID: \"d065f53f-cb94-43a3-9625-dd5bac8bc2f9\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss"
Sep 19 01:51:00.740359 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:00.740196    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d065f53f-cb94-43a3-9625-dd5bac8bc2f9-openshift-service-ca\") pod \"maas-api-5598d8c9fb-bbkss\" (UID: \"d065f53f-cb94-43a3-9625-dd5bac8bc2f9\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss"
Sep 19 01:51:00.740359 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:00.740232    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d065f53f-cb94-43a3-9625-dd5bac8bc2f9-maas-api-tls\") pod \"maas-api-5598d8c9fb-bbkss\" (UID: \"d065f53f-cb94-43a3-9625-dd5bac8bc2f9\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss"
Sep 19 01:51:00.740359 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:00.740293    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2n55\" (UniqueName: \"kubernetes.io/projected/d065f53f-cb94-43a3-9625-dd5bac8bc2f9-kube-api-access-m2n55\") pod \"maas-api-5598d8c9fb-bbkss\" (UID: \"d065f53f-cb94-43a3-9625-dd5bac8bc2f9\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss"
Sep 19 01:51:00.841757 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:00.841718    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d065f53f-cb94-43a3-9625-dd5bac8bc2f9-metrics-certs\") pod \"maas-api-5598d8c9fb-bbkss\" (UID: \"d065f53f-cb94-43a3-9625-dd5bac8bc2f9\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss"
Sep 19 01:51:00.842012 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:00.841764    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d065f53f-cb94-43a3-9625-dd5bac8bc2f9-openshift-service-ca\") pod \"maas-api-5598d8c9fb-bbkss\" (UID: \"d065f53f-cb94-43a3-9625-dd5bac8bc2f9\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss"
Sep 19 01:51:00.842012 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:00.841796    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d065f53f-cb94-43a3-9625-dd5bac8bc2f9-maas-api-tls\") pod \"maas-api-5598d8c9fb-bbkss\" (UID: \"d065f53f-cb94-43a3-9625-dd5bac8bc2f9\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss"
Sep 19 01:51:00.842012 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:00.841867    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2n55\" (UniqueName: \"kubernetes.io/projected/d065f53f-cb94-43a3-9625-dd5bac8bc2f9-kube-api-access-m2n55\") pod \"maas-api-5598d8c9fb-bbkss\" (UID: \"d065f53f-cb94-43a3-9625-dd5bac8bc2f9\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss"
Sep 19 01:51:00.842693 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:00.842658    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d065f53f-cb94-43a3-9625-dd5bac8bc2f9-openshift-service-ca\") pod \"maas-api-5598d8c9fb-bbkss\" (UID: \"d065f53f-cb94-43a3-9625-dd5bac8bc2f9\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss"
Sep 19 01:51:00.844442 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:00.844422    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d065f53f-cb94-43a3-9625-dd5bac8bc2f9-metrics-certs\") pod \"maas-api-5598d8c9fb-bbkss\" (UID: \"d065f53f-cb94-43a3-9625-dd5bac8bc2f9\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss"
Sep 19 01:51:00.844523 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:00.844455    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d065f53f-cb94-43a3-9625-dd5bac8bc2f9-maas-api-tls\") pod \"maas-api-5598d8c9fb-bbkss\" (UID: \"d065f53f-cb94-43a3-9625-dd5bac8bc2f9\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss"
Sep 19 01:51:00.850999 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:00.850972    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2n55\" (UniqueName: \"kubernetes.io/projected/d065f53f-cb94-43a3-9625-dd5bac8bc2f9-kube-api-access-m2n55\") pod \"maas-api-5598d8c9fb-bbkss\" (UID: \"d065f53f-cb94-43a3-9625-dd5bac8bc2f9\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss"
Sep 19 01:51:00.929835 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:00.929748    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss"
Sep 19 01:51:01.209598 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:01.209570    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss"]
Sep 19 01:51:01.210998 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:51:01.210971    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd065f53f_cb94_43a3_9625_dd5bac8bc2f9.slice/crio-5dda0e2b3822b231195a11aab84ca3bf63b34075dc71e7946d8491aa1aed82f7 WatchSource:0}: Error finding container 5dda0e2b3822b231195a11aab84ca3bf63b34075dc71e7946d8491aa1aed82f7: Status 404 returned error can't find the container with id 5dda0e2b3822b231195a11aab84ca3bf63b34075dc71e7946d8491aa1aed82f7
Sep 19 01:51:01.212406 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:01.212384    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 01:51:01.739985 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:01.739954    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss" event={"ID":"d065f53f-cb94-43a3-9625-dd5bac8bc2f9","Type":"ContainerStarted","Data":"5dda0e2b3822b231195a11aab84ca3bf63b34075dc71e7946d8491aa1aed82f7"}
Sep 19 01:51:02.748150 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:02.748114    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss" event={"ID":"d065f53f-cb94-43a3-9625-dd5bac8bc2f9","Type":"ContainerStarted","Data":"8f2662c28939934ccbe3b596e4ca6e609836a756fccb0ea49bfffeea4cbc84d0"}
Sep 19 01:51:02.748671 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:02.748248    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss"
Sep 19 01:51:02.767013 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:02.766963    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss" podStartSLOduration=2.292182355 podStartE2EDuration="2.766947127s" totalImagesPullingTime="474.764772ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:51:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:51:02.764694625 +0000 UTC m=+2317.671556682" watchObservedRunningTime="2026-09-19 01:51:02.766947127 +0000 UTC m=+2317.673809175"
Sep 19 01:51:07.873529 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:07.873483    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss"]
Sep 19 01:51:07.873936 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:07.873690    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss" podUID="d065f53f-cb94-43a3-9625-dd5bac8bc2f9" containerName="maas-api" containerID="cri-o://8f2662c28939934ccbe3b596e4ca6e609836a756fccb0ea49bfffeea4cbc84d0" gracePeriod=30
Sep 19 01:51:07.880261 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:07.880233    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss"
Sep 19 01:51:07.936637 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:07.936604    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-68c86dfd56-dv2rv"]
Sep 19 01:51:07.940106 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:07.940089    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-dv2rv"
Sep 19 01:51:07.959368 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:07.959340    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-68c86dfd56-dv2rv"]
Sep 19 01:51:08.010964 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.010930    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67xlv\" (UniqueName: \"kubernetes.io/projected/943ce655-9f6f-4cf3-b573-db505c807c43-kube-api-access-67xlv\") pod \"maas-api-68c86dfd56-dv2rv\" (UID: \"943ce655-9f6f-4cf3-b573-db505c807c43\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-dv2rv"
Sep 19 01:51:08.011166 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.010975    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/943ce655-9f6f-4cf3-b573-db505c807c43-metrics-certs\") pod \"maas-api-68c86dfd56-dv2rv\" (UID: \"943ce655-9f6f-4cf3-b573-db505c807c43\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-dv2rv"
Sep 19 01:51:08.011166 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.011006    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/943ce655-9f6f-4cf3-b573-db505c807c43-maas-api-tls\") pod \"maas-api-68c86dfd56-dv2rv\" (UID: \"943ce655-9f6f-4cf3-b573-db505c807c43\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-dv2rv"
Sep 19 01:51:08.011166 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.011048    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/943ce655-9f6f-4cf3-b573-db505c807c43-openshift-service-ca\") pod \"maas-api-68c86dfd56-dv2rv\" (UID: \"943ce655-9f6f-4cf3-b573-db505c807c43\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-dv2rv"
Sep 19 01:51:08.112486 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.112457    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/943ce655-9f6f-4cf3-b573-db505c807c43-metrics-certs\") pod \"maas-api-68c86dfd56-dv2rv\" (UID: \"943ce655-9f6f-4cf3-b573-db505c807c43\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-dv2rv"
Sep 19 01:51:08.112638 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.112523    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/943ce655-9f6f-4cf3-b573-db505c807c43-maas-api-tls\") pod \"maas-api-68c86dfd56-dv2rv\" (UID: \"943ce655-9f6f-4cf3-b573-db505c807c43\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-dv2rv"
Sep 19 01:51:08.112638 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.112583    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/943ce655-9f6f-4cf3-b573-db505c807c43-openshift-service-ca\") pod \"maas-api-68c86dfd56-dv2rv\" (UID: \"943ce655-9f6f-4cf3-b573-db505c807c43\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-dv2rv"
Sep 19 01:51:08.112775 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.112675    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67xlv\" (UniqueName: \"kubernetes.io/projected/943ce655-9f6f-4cf3-b573-db505c807c43-kube-api-access-67xlv\") pod \"maas-api-68c86dfd56-dv2rv\" (UID: \"943ce655-9f6f-4cf3-b573-db505c807c43\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-dv2rv"
Sep 19 01:51:08.113554 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.113528    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/943ce655-9f6f-4cf3-b573-db505c807c43-openshift-service-ca\") pod \"maas-api-68c86dfd56-dv2rv\" (UID: \"943ce655-9f6f-4cf3-b573-db505c807c43\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-dv2rv"
Sep 19 01:51:08.114926 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.114910    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/943ce655-9f6f-4cf3-b573-db505c807c43-maas-api-tls\") pod \"maas-api-68c86dfd56-dv2rv\" (UID: \"943ce655-9f6f-4cf3-b573-db505c807c43\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-dv2rv"
Sep 19 01:51:08.115133 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.115112    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/943ce655-9f6f-4cf3-b573-db505c807c43-metrics-certs\") pod \"maas-api-68c86dfd56-dv2rv\" (UID: \"943ce655-9f6f-4cf3-b573-db505c807c43\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-dv2rv"
Sep 19 01:51:08.122536 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.122514    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67xlv\" (UniqueName: \"kubernetes.io/projected/943ce655-9f6f-4cf3-b573-db505c807c43-kube-api-access-67xlv\") pod \"maas-api-68c86dfd56-dv2rv\" (UID: \"943ce655-9f6f-4cf3-b573-db505c807c43\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-dv2rv"
Sep 19 01:51:08.142373 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.142315    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss"
Sep 19 01:51:08.213417 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.213386    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d065f53f-cb94-43a3-9625-dd5bac8bc2f9-openshift-service-ca\") pod \"d065f53f-cb94-43a3-9625-dd5bac8bc2f9\" (UID: \"d065f53f-cb94-43a3-9625-dd5bac8bc2f9\") "
Sep 19 01:51:08.213417 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.213419    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2n55\" (UniqueName: \"kubernetes.io/projected/d065f53f-cb94-43a3-9625-dd5bac8bc2f9-kube-api-access-m2n55\") pod \"d065f53f-cb94-43a3-9625-dd5bac8bc2f9\" (UID: \"d065f53f-cb94-43a3-9625-dd5bac8bc2f9\") "
Sep 19 01:51:08.213652 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.213461    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d065f53f-cb94-43a3-9625-dd5bac8bc2f9-metrics-certs\") pod \"d065f53f-cb94-43a3-9625-dd5bac8bc2f9\" (UID: \"d065f53f-cb94-43a3-9625-dd5bac8bc2f9\") "
Sep 19 01:51:08.213652 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.213485    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d065f53f-cb94-43a3-9625-dd5bac8bc2f9-maas-api-tls\") pod \"d065f53f-cb94-43a3-9625-dd5bac8bc2f9\" (UID: \"d065f53f-cb94-43a3-9625-dd5bac8bc2f9\") "
Sep 19 01:51:08.213854 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.213828    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d065f53f-cb94-43a3-9625-dd5bac8bc2f9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d065f53f-cb94-43a3-9625-dd5bac8bc2f9" (UID: "d065f53f-cb94-43a3-9625-dd5bac8bc2f9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 01:51:08.215644 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.215622    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d065f53f-cb94-43a3-9625-dd5bac8bc2f9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d065f53f-cb94-43a3-9625-dd5bac8bc2f9" (UID: "d065f53f-cb94-43a3-9625-dd5bac8bc2f9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:51:08.215723 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.215638    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d065f53f-cb94-43a3-9625-dd5bac8bc2f9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d065f53f-cb94-43a3-9625-dd5bac8bc2f9" (UID: "d065f53f-cb94-43a3-9625-dd5bac8bc2f9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 01:51:08.215723 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.215640    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d065f53f-cb94-43a3-9625-dd5bac8bc2f9-kube-api-access-m2n55" (OuterVolumeSpecName: "kube-api-access-m2n55") pod "d065f53f-cb94-43a3-9625-dd5bac8bc2f9" (UID: "d065f53f-cb94-43a3-9625-dd5bac8bc2f9"). InnerVolumeSpecName "kube-api-access-m2n55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 01:51:08.264977 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.264950    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-dv2rv"
Sep 19 01:51:08.315173 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.315144    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d065f53f-cb94-43a3-9625-dd5bac8bc2f9-openshift-service-ca\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:51:08.315173 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.315174    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2n55\" (UniqueName: \"kubernetes.io/projected/d065f53f-cb94-43a3-9625-dd5bac8bc2f9-kube-api-access-m2n55\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:51:08.315362 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.315190    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d065f53f-cb94-43a3-9625-dd5bac8bc2f9-metrics-certs\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:51:08.315362 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.315201    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d065f53f-cb94-43a3-9625-dd5bac8bc2f9-maas-api-tls\") on node \"ip-10-0-131-76.ec2.internal\" DevicePath \"\""
Sep 19 01:51:08.544063 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.544037    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-68c86dfd56-dv2rv"]
Sep 19 01:51:08.545193 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:51:08.545169    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod943ce655_9f6f_4cf3_b573_db505c807c43.slice/crio-b345aacd388c85fe0a355fb9f55fa20bba960cea0dfe9992bc5e51c8676791d7 WatchSource:0}: Error finding container b345aacd388c85fe0a355fb9f55fa20bba960cea0dfe9992bc5e51c8676791d7: Status 404 returned error can't find the container with id b345aacd388c85fe0a355fb9f55fa20bba960cea0dfe9992bc5e51c8676791d7
Sep 19 01:51:08.784182 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.784142    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-dv2rv" event={"ID":"943ce655-9f6f-4cf3-b573-db505c807c43","Type":"ContainerStarted","Data":"b345aacd388c85fe0a355fb9f55fa20bba960cea0dfe9992bc5e51c8676791d7"}
Sep 19 01:51:08.785511 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.785473    2580 generic.go:356] "Generic (PLEG): container finished" podID="d065f53f-cb94-43a3-9625-dd5bac8bc2f9" containerID="8f2662c28939934ccbe3b596e4ca6e609836a756fccb0ea49bfffeea4cbc84d0" exitCode=0
Sep 19 01:51:08.785637 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.785540    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss" event={"ID":"d065f53f-cb94-43a3-9625-dd5bac8bc2f9","Type":"ContainerDied","Data":"8f2662c28939934ccbe3b596e4ca6e609836a756fccb0ea49bfffeea4cbc84d0"}
Sep 19 01:51:08.785637 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.785577    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss" event={"ID":"d065f53f-cb94-43a3-9625-dd5bac8bc2f9","Type":"ContainerDied","Data":"5dda0e2b3822b231195a11aab84ca3bf63b34075dc71e7946d8491aa1aed82f7"}
Sep 19 01:51:08.785637 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.785598    2580 scope.go:117] "RemoveContainer" containerID="8f2662c28939934ccbe3b596e4ca6e609836a756fccb0ea49bfffeea4cbc84d0"
Sep 19 01:51:08.785637 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.785541    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss"
Sep 19 01:51:08.795291 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.795276    2580 scope.go:117] "RemoveContainer" containerID="8f2662c28939934ccbe3b596e4ca6e609836a756fccb0ea49bfffeea4cbc84d0"
Sep 19 01:51:08.795575 ip-10-0-131-76 kubenswrapper[2580]: E0919 01:51:08.795558    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f2662c28939934ccbe3b596e4ca6e609836a756fccb0ea49bfffeea4cbc84d0\": container with ID starting with 8f2662c28939934ccbe3b596e4ca6e609836a756fccb0ea49bfffeea4cbc84d0 not found: ID does not exist" containerID="8f2662c28939934ccbe3b596e4ca6e609836a756fccb0ea49bfffeea4cbc84d0"
Sep 19 01:51:08.795639 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.795585    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f2662c28939934ccbe3b596e4ca6e609836a756fccb0ea49bfffeea4cbc84d0"} err="failed to get container status \"8f2662c28939934ccbe3b596e4ca6e609836a756fccb0ea49bfffeea4cbc84d0\": rpc error: code = NotFound desc = could not find container \"8f2662c28939934ccbe3b596e4ca6e609836a756fccb0ea49bfffeea4cbc84d0\": container with ID starting with 8f2662c28939934ccbe3b596e4ca6e609836a756fccb0ea49bfffeea4cbc84d0 not found: ID does not exist"
Sep 19 01:51:08.812385 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.812276    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss"]
Sep 19 01:51:08.815004 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:08.814984    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-bbkss"]
Sep 19 01:51:09.064521 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:09.064425    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d065f53f-cb94-43a3-9625-dd5bac8bc2f9" path="/var/lib/kubelet/pods/d065f53f-cb94-43a3-9625-dd5bac8bc2f9/volumes"
Sep 19 01:51:09.794275 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:09.794241    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-dv2rv" event={"ID":"943ce655-9f6f-4cf3-b573-db505c807c43","Type":"ContainerStarted","Data":"3309717dc16ca1746ffbf4c46acc1491fdfb0b5358be9d3174b1711b9aed6e69"}
Sep 19 01:51:09.794435 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:09.794347    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-dv2rv"
Sep 19 01:51:09.820468 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:09.820415    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-dv2rv" podStartSLOduration=2.540700866 podStartE2EDuration="2.820401208s" totalImagesPullingTime="279.700342ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:51:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:51:09.819241204 +0000 UTC m=+2324.726103253" watchObservedRunningTime="2026-09-19 01:51:09.820401208 +0000 UTC m=+2324.727263256"
Sep 19 01:51:15.806769 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:15.806740    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-dv2rv"
Sep 19 01:51:35.800387 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:35.800354    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-699c784c97-hc98f_258a7e21-2f52-4701-a11d-d98fa18aa20f/authorino/0.log"
Sep 19 01:51:41.343068 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:41.343038    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-45rlh_1e5626e3-1d2c-470c-b619-17ec423c7922/manager/0.log"
Sep 19 01:51:41.468200 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:41.468158    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-cl6v6_74af2f71-487e-406c-b594-3292cde8ea44/manager/0.log"
Sep 19 01:51:41.602334 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:41.602255    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5bc8cb64b6-8cqf4_436955fc-9575-4cb2-bcdc-dec43a78bec6/manager/0.log"
Sep 19 01:51:41.733026 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:41.732996    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-d9vhq_e7192a01-988e-4143-a5bf-37ac7910fcd4/server/0.log"
Sep 19 01:51:41.983150 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:41.983122    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-748475db69-5dq5z_2015c019-405a-4565-9c96-8ddda261051c/copy-manifests/0.log"
Sep 19 01:51:41.992392 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:41.992371    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-748475db69-5dq5z_2015c019-405a-4565-9c96-8ddda261051c/manager/0.log"
Sep 19 01:51:42.744951 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:42.744916    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-68c86dfd56-dv2rv_943ce655-9f6f-4cf3-b573-db505c807c43/maas-api/0.log"
Sep 19 01:51:43.084398 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:43.084314    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-fcpqw_aaf5f974-69f5-43a7-99c9-52f66932c8db/postgres/0.log"
Sep 19 01:51:43.895444 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:43.895410    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-699c784c97-hc98f_258a7e21-2f52-4701-a11d-d98fa18aa20f/authorino/0.log"
Sep 19 01:51:44.016203 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:44.016165    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-rmhw5_4370cd9a-5066-4f21-bbc9-620ecfdcd6ff/manager/0.log"
Sep 19 01:51:44.136608 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:44.136572    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-zgxhm_5ca3e984-22f7-4867-8d0b-0c61c8d9c6cf/manager/0.log"
Sep 19 01:51:44.390884 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:44.390850    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-cpr4n_2ed2e6bc-763f-474e-8eb5-0e730b7ce45d/manager/0.log"
Sep 19 01:51:44.649011 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:44.648931    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-svhp6_9224c100-d688-4875-977f-95680c0778ac/manager/0.log"
Sep 19 01:51:45.363077 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:45.363046    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c6796f956-wzrsb_db018a41-4085-448a-bcae-81fe88b9f53a/kube-auth-proxy/0.log"
Sep 19 01:51:45.620397 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:45.620316    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-bb6qh_ee7c3e3f-b988-4467-82b7-6bda62a1140d/payload-pre-processing/0.log"
Sep 19 01:51:45.751401 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:45.751361    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-42wkt_c9ba5520-238c-4d7b-9266-16a1c0f76e8d/payload-processing/0.log"
Sep 19 01:51:45.877045 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:45.876968    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69d68c9d66-2gblx_9cb55096-dad6-4d87-a63b-48e18dc7a011/router/0.log"
Sep 19 01:51:52.760021 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:52.759990    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vc5k7/must-gather-fps7f"]
Sep 19 01:51:52.760471 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:52.760456    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d065f53f-cb94-43a3-9625-dd5bac8bc2f9" containerName="maas-api"
Sep 19 01:51:52.760529 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:52.760472    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d065f53f-cb94-43a3-9625-dd5bac8bc2f9" containerName="maas-api"
Sep 19 01:51:52.760568 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:52.760547    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d065f53f-cb94-43a3-9625-dd5bac8bc2f9" containerName="maas-api"
Sep 19 01:51:52.765237 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:52.765217    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vc5k7/must-gather-fps7f"
Sep 19 01:51:52.768185 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:52.768157    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vc5k7\"/\"default-dockercfg-gbbnr\""
Sep 19 01:51:52.768298 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:52.768195    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vc5k7\"/\"kube-root-ca.crt\""
Sep 19 01:51:52.768298 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:52.768196    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vc5k7\"/\"openshift-service-ca.crt\""
Sep 19 01:51:52.778247 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:52.778224    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vc5k7/must-gather-fps7f"]
Sep 19 01:51:52.916228 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:52.916193    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr5f9\" (UniqueName: \"kubernetes.io/projected/c6674593-1bdb-471d-a172-78390d4860e6-kube-api-access-pr5f9\") pod \"must-gather-fps7f\" (UID: \"c6674593-1bdb-471d-a172-78390d4860e6\") " pod="openshift-must-gather-vc5k7/must-gather-fps7f"
Sep 19 01:51:52.916390 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:52.916307    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c6674593-1bdb-471d-a172-78390d4860e6-must-gather-output\") pod \"must-gather-fps7f\" (UID: \"c6674593-1bdb-471d-a172-78390d4860e6\") " pod="openshift-must-gather-vc5k7/must-gather-fps7f"
Sep 19 01:51:53.017431 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:53.017350    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr5f9\" (UniqueName: \"kubernetes.io/projected/c6674593-1bdb-471d-a172-78390d4860e6-kube-api-access-pr5f9\") pod \"must-gather-fps7f\" (UID: \"c6674593-1bdb-471d-a172-78390d4860e6\") " pod="openshift-must-gather-vc5k7/must-gather-fps7f"
Sep 19 01:51:53.017431 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:53.017428    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c6674593-1bdb-471d-a172-78390d4860e6-must-gather-output\") pod \"must-gather-fps7f\" (UID: \"c6674593-1bdb-471d-a172-78390d4860e6\") " pod="openshift-must-gather-vc5k7/must-gather-fps7f"
Sep 19 01:51:53.017742 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:53.017727    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c6674593-1bdb-471d-a172-78390d4860e6-must-gather-output\") pod \"must-gather-fps7f\" (UID: \"c6674593-1bdb-471d-a172-78390d4860e6\") " pod="openshift-must-gather-vc5k7/must-gather-fps7f"
Sep 19 01:51:53.027260 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:53.027238    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr5f9\" (UniqueName: \"kubernetes.io/projected/c6674593-1bdb-471d-a172-78390d4860e6-kube-api-access-pr5f9\") pod \"must-gather-fps7f\" (UID: \"c6674593-1bdb-471d-a172-78390d4860e6\") " pod="openshift-must-gather-vc5k7/must-gather-fps7f"
Sep 19 01:51:53.075913 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:53.075888    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vc5k7/must-gather-fps7f"
Sep 19 01:51:53.357542 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:53.357515    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vc5k7/must-gather-fps7f"]
Sep 19 01:51:53.359373 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:51:53.359347    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6674593_1bdb_471d_a172_78390d4860e6.slice/crio-158d390794bd997a84cec8e9fd4551f41ead9b1582a974545c5fb3591d5cc9cc WatchSource:0}: Error finding container 158d390794bd997a84cec8e9fd4551f41ead9b1582a974545c5fb3591d5cc9cc: Status 404 returned error can't find the container with id 158d390794bd997a84cec8e9fd4551f41ead9b1582a974545c5fb3591d5cc9cc
Sep 19 01:51:54.062024 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:54.061988    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vc5k7/must-gather-fps7f" event={"ID":"c6674593-1bdb-471d-a172-78390d4860e6","Type":"ContainerStarted","Data":"158d390794bd997a84cec8e9fd4551f41ead9b1582a974545c5fb3591d5cc9cc"}
Sep 19 01:51:55.073392 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:55.073346    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vc5k7/must-gather-fps7f" event={"ID":"c6674593-1bdb-471d-a172-78390d4860e6","Type":"ContainerStarted","Data":"5cb48e81d81bdf1b9c904881f4dad1e9786b9ad5438d2af571f42711880b9529"}
Sep 19 01:51:55.073898 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:55.073408    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vc5k7/must-gather-fps7f" event={"ID":"c6674593-1bdb-471d-a172-78390d4860e6","Type":"ContainerStarted","Data":"d79934f04c496e6e2d05207b547212594274d679a887d05c967201f911aafd95"}
Sep 19 01:51:55.092474 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:55.092424    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vc5k7/must-gather-fps7f" podStartSLOduration=2.377970582 podStartE2EDuration="3.092408217s" totalImagesPullingTime="714.437635ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 01:51:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:51:55.088660544 +0000 UTC m=+2369.995522591" watchObservedRunningTime="2026-09-19 01:51:55.092408217 +0000 UTC m=+2369.999270265"
Sep 19 01:51:55.773810 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:55.773775    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-kj96s_2cd09385-9d78-428c-bb46-efeb37e8e55a/global-pull-secret-syncer/0.log"
Sep 19 01:51:55.953176 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:55.953145    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9n69k_2f997c70-10b2-49e0-8711-00e914baa6c4/konnectivity-agent/0.log"
Sep 19 01:51:55.998848 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:51:55.998819    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-76.ec2.internal_0034bf5044f034aee1efd2c2a680ecba/haproxy/0.log"
Sep 19 01:52:00.836684 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:00.835767    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-699c784c97-hc98f_258a7e21-2f52-4701-a11d-d98fa18aa20f/authorino/0.log"
Sep 19 01:52:00.885115 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:00.885084    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-rmhw5_4370cd9a-5066-4f21-bbc9-620ecfdcd6ff/manager/0.log"
Sep 19 01:52:00.948437 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:00.948279    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-zgxhm_5ca3e984-22f7-4867-8d0b-0c61c8d9c6cf/manager/0.log"
Sep 19 01:52:01.066406 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:01.066367    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-cpr4n_2ed2e6bc-763f-474e-8eb5-0e730b7ce45d/manager/0.log"
Sep 19 01:52:01.247536 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:01.246972    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-svhp6_9224c100-d688-4875-977f-95680c0778ac/manager/0.log"
Sep 19 01:52:03.270029 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:03.269983    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-847b8bd8d7-9bmsn_c63ea3c5-742f-4077-a661-388d76e45d63/cluster-monitoring-operator/0.log"
Sep 19 01:52:03.301217 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:03.300271    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-5q7kn_551bcebb-cd3c-4ca5-ac56-54c6551248b0/kube-state-metrics/0.log"
Sep 19 01:52:03.329643 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:03.329608    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-5q7kn_551bcebb-cd3c-4ca5-ac56-54c6551248b0/kube-rbac-proxy-main/0.log"
Sep 19 01:52:03.351264 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:03.351234    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-5q7kn_551bcebb-cd3c-4ca5-ac56-54c6551248b0/kube-rbac-proxy-self/0.log"
Sep 19 01:52:03.452820 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:03.452792    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8kpmp_e43af3ba-8b7a-488a-98e1-0acd8e823a1f/node-exporter/0.log"
Sep 19 01:52:03.478673 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:03.478631    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8kpmp_e43af3ba-8b7a-488a-98e1-0acd8e823a1f/kube-rbac-proxy/0.log"
Sep 19 01:52:03.499865 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:03.499836    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8kpmp_e43af3ba-8b7a-488a-98e1-0acd8e823a1f/init-textfile/0.log"
Sep 19 01:52:03.694124 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:03.694042    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-86gqf_31e2d5af-7ffd-4089-8628-eca0941659a9/kube-rbac-proxy-main/0.log"
Sep 19 01:52:03.722239 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:03.722213    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-86gqf_31e2d5af-7ffd-4089-8628-eca0941659a9/kube-rbac-proxy-self/0.log"
Sep 19 01:52:03.747783 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:03.747746    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-86gqf_31e2d5af-7ffd-4089-8628-eca0941659a9/openshift-state-metrics/0.log"
Sep 19 01:52:03.973236 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:03.973146    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-5ld7x_b3c58358-5b1f-4617-9945-42246f59e5f7/prometheus-operator/0.log"
Sep 19 01:52:03.995157 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:03.995101    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-5ld7x_b3c58358-5b1f-4617-9945-42246f59e5f7/kube-rbac-proxy/0.log"
Sep 19 01:52:04.133151 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:04.133116    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-688f75fbf5-kvmhp_8c057f6b-b862-40fa-b1dd-deb4c2102eed/thanos-query/0.log"
Sep 19 01:52:04.156758 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:04.156720    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-688f75fbf5-kvmhp_8c057f6b-b862-40fa-b1dd-deb4c2102eed/kube-rbac-proxy-web/0.log"
Sep 19 01:52:04.179648 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:04.179610    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-688f75fbf5-kvmhp_8c057f6b-b862-40fa-b1dd-deb4c2102eed/kube-rbac-proxy/0.log"
Sep 19 01:52:04.210170 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:04.210124    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-688f75fbf5-kvmhp_8c057f6b-b862-40fa-b1dd-deb4c2102eed/prom-label-proxy/0.log"
Sep 19 01:52:04.233789 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:04.233752    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-688f75fbf5-kvmhp_8c057f6b-b862-40fa-b1dd-deb4c2102eed/kube-rbac-proxy-rules/0.log"
Sep 19 01:52:04.256042 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:04.256013    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-688f75fbf5-kvmhp_8c057f6b-b862-40fa-b1dd-deb4c2102eed/kube-rbac-proxy-metrics/0.log"
Sep 19 01:52:05.751948 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:05.751908    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg"]
Sep 19 01:52:05.760828 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:05.760801    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg"
Sep 19 01:52:05.766450 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:05.766422    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg"]
Sep 19 01:52:05.872914 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:05.872865    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c199ee83-0d04-4969-a2b3-0522ede45217-proc\") pod \"perf-node-gather-daemonset-vcmsg\" (UID: \"c199ee83-0d04-4969-a2b3-0522ede45217\") " pod="openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg"
Sep 19 01:52:05.873263 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:05.873236    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c199ee83-0d04-4969-a2b3-0522ede45217-lib-modules\") pod \"perf-node-gather-daemonset-vcmsg\" (UID: \"c199ee83-0d04-4969-a2b3-0522ede45217\") " pod="openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg"
Sep 19 01:52:05.873524 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:05.873474    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c199ee83-0d04-4969-a2b3-0522ede45217-sys\") pod \"perf-node-gather-daemonset-vcmsg\" (UID: \"c199ee83-0d04-4969-a2b3-0522ede45217\") " pod="openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg"
Sep 19 01:52:05.873634 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:05.873571    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c199ee83-0d04-4969-a2b3-0522ede45217-podres\") pod \"perf-node-gather-daemonset-vcmsg\" (UID: \"c199ee83-0d04-4969-a2b3-0522ede45217\") " pod="openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg"
Sep 19 01:52:05.873634 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:05.873622    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cfq7\" (UniqueName: \"kubernetes.io/projected/c199ee83-0d04-4969-a2b3-0522ede45217-kube-api-access-5cfq7\") pod \"perf-node-gather-daemonset-vcmsg\" (UID: \"c199ee83-0d04-4969-a2b3-0522ede45217\") " pod="openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg"
Sep 19 01:52:05.975069 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:05.975029    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c199ee83-0d04-4969-a2b3-0522ede45217-sys\") pod \"perf-node-gather-daemonset-vcmsg\" (UID: \"c199ee83-0d04-4969-a2b3-0522ede45217\") " pod="openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg"
Sep 19 01:52:05.975472 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:05.975454    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c199ee83-0d04-4969-a2b3-0522ede45217-podres\") pod \"perf-node-gather-daemonset-vcmsg\" (UID: \"c199ee83-0d04-4969-a2b3-0522ede45217\") " pod="openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg"
Sep 19 01:52:05.975670 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:05.975655    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cfq7\" (UniqueName: \"kubernetes.io/projected/c199ee83-0d04-4969-a2b3-0522ede45217-kube-api-access-5cfq7\") pod \"perf-node-gather-daemonset-vcmsg\" (UID: \"c199ee83-0d04-4969-a2b3-0522ede45217\") " pod="openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg"
Sep 19 01:52:05.975758 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:05.975727    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c199ee83-0d04-4969-a2b3-0522ede45217-podres\") pod \"perf-node-gather-daemonset-vcmsg\" (UID: \"c199ee83-0d04-4969-a2b3-0522ede45217\") " pod="openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg"
Sep 19 01:52:05.975998 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:05.975336    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c199ee83-0d04-4969-a2b3-0522ede45217-sys\") pod \"perf-node-gather-daemonset-vcmsg\" (UID: \"c199ee83-0d04-4969-a2b3-0522ede45217\") " pod="openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg"
Sep 19 01:52:05.976120 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:05.975907    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c199ee83-0d04-4969-a2b3-0522ede45217-proc\") pod \"perf-node-gather-daemonset-vcmsg\" (UID: \"c199ee83-0d04-4969-a2b3-0522ede45217\") " pod="openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg"
Sep 19 01:52:05.976253 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:05.976240    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c199ee83-0d04-4969-a2b3-0522ede45217-lib-modules\") pod \"perf-node-gather-daemonset-vcmsg\" (UID: \"c199ee83-0d04-4969-a2b3-0522ede45217\") " pod="openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg"
Sep 19 01:52:05.976530 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:05.975958    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c199ee83-0d04-4969-a2b3-0522ede45217-proc\") pod \"perf-node-gather-daemonset-vcmsg\" (UID: \"c199ee83-0d04-4969-a2b3-0522ede45217\") " pod="openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg"
Sep 19 01:52:05.976636 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:05.976434    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c199ee83-0d04-4969-a2b3-0522ede45217-lib-modules\") pod \"perf-node-gather-daemonset-vcmsg\" (UID: \"c199ee83-0d04-4969-a2b3-0522ede45217\") " pod="openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg"
Sep 19 01:52:05.984653 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:05.984630    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cfq7\" (UniqueName: \"kubernetes.io/projected/c199ee83-0d04-4969-a2b3-0522ede45217-kube-api-access-5cfq7\") pod \"perf-node-gather-daemonset-vcmsg\" (UID: \"c199ee83-0d04-4969-a2b3-0522ede45217\") " pod="openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg"
Sep 19 01:52:06.086568 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:06.086458    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg"
Sep 19 01:52:06.129948 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:06.129793    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f6bdh_74c8e2f6-1b01-4bd1-b491-ad407a15a109/console-operator/1.log"
Sep 19 01:52:06.138969 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:06.138940    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f6bdh_74c8e2f6-1b01-4bd1-b491-ad407a15a109/console-operator/2.log"
Sep 19 01:52:06.379435 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:06.379403    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg"]
Sep 19 01:52:06.381301 ip-10-0-131-76 kubenswrapper[2580]: W0919 01:52:06.381275    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc199ee83_0d04_4969_a2b3_0522ede45217.slice/crio-a6e99e322253cfd6b0639d4d9d12d7be35110daa1cf3612e6ea460a43936ec4b WatchSource:0}: Error finding container a6e99e322253cfd6b0639d4d9d12d7be35110daa1cf3612e6ea460a43936ec4b: Status 404 returned error can't find the container with id a6e99e322253cfd6b0639d4d9d12d7be35110daa1cf3612e6ea460a43936ec4b
Sep 19 01:52:06.710012 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:06.709931    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-56c5d49fcb-2dtxc_1917c1fa-4d49-4356-b2ee-3f729d8866e4/console/0.log"
Sep 19 01:52:07.191761 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:07.191723    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg" event={"ID":"c199ee83-0d04-4969-a2b3-0522ede45217","Type":"ContainerStarted","Data":"b1abffa2c53266bb681c562574c1a2f1f61766ba97f886fcb2d973fb0bfc3ad0"}
Sep 19 01:52:07.192191 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:07.191786    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg" event={"ID":"c199ee83-0d04-4969-a2b3-0522ede45217","Type":"ContainerStarted","Data":"a6e99e322253cfd6b0639d4d9d12d7be35110daa1cf3612e6ea460a43936ec4b"}
Sep 19 01:52:07.192191 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:07.191842    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg"
Sep 19 01:52:07.210261 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:07.210214    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg" podStartSLOduration=2.210180101 podStartE2EDuration="2.210180101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 01:52:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 01:52:07.20718511 +0000 UTC m=+2382.114047161" watchObservedRunningTime="2026-09-19 01:52:07.210180101 +0000 UTC m=+2382.117042150"
Sep 19 01:52:07.445560 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:07.445437    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5c4db8bdf9-zmkjg_73e4b372-92e4-4eeb-ac37-6c5ec11217f0/volume-data-source-validator/0.log"
Sep 19 01:52:08.525839 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:08.525806    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v6ppc_f921d091-7bf8-4d6d-81e4-6162afac0198/dns/0.log"
Sep 19 01:52:08.546423 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:08.546395    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v6ppc_f921d091-7bf8-4d6d-81e4-6162afac0198/kube-rbac-proxy/0.log"
Sep 19 01:52:08.638519 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:08.638459    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-grkk4_54557d74-f7ed-44ac-b483-9942019a9c4d/dns-node-resolver/0.log"
Sep 19 01:52:09.239534 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:09.239482    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dvd6m_2857e44c-032d-4586-b438-26ad712aef9c/node-ca/0.log"
Sep 19 01:52:11.216138 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:11.216101    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c6796f956-wzrsb_db018a41-4085-448a-bcae-81fe88b9f53a/kube-auth-proxy/0.log"
Sep 19 01:52:11.293788 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:11.293753    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-bb6qh_ee7c3e3f-b988-4467-82b7-6bda62a1140d/payload-pre-processing/0.log"
Sep 19 01:52:11.340319 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:11.340284    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-42wkt_c9ba5520-238c-4d7b-9266-16a1c0f76e8d/payload-processing/0.log"
Sep 19 01:52:11.364215 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:11.364183    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69d68c9d66-2gblx_9cb55096-dad6-4d87-a63b-48e18dc7a011/router/0.log"
Sep 19 01:52:12.324338 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:12.324298    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p4cpb_aa9e1f35-1684-4704-8bd7-532245d3fc97/serve-healthcheck-canary/0.log"
Sep 19 01:52:12.838257 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:12.838220    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-p6l4m_39355481-6d16-4553-961a-f03ca171e712/insights-operator/0.log"
Sep 19 01:52:12.839034 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:12.839004    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-p6l4m_39355481-6d16-4553-961a-f03ca171e712/insights-operator/1.log"
Sep 19 01:52:12.996754 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:12.996719    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mdz52_8830a56c-b4f4-4bbc-9303-52cd0b23d620/kube-rbac-proxy/0.log"
Sep 19 01:52:13.033562 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:13.033526    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mdz52_8830a56c-b4f4-4bbc-9303-52cd0b23d620/exporter/0.log"
Sep 19 01:52:13.055745 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:13.055716    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mdz52_8830a56c-b4f4-4bbc-9303-52cd0b23d620/extractor/0.log"
Sep 19 01:52:13.211667 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:13.211589    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vc5k7/perf-node-gather-daemonset-vcmsg"
Sep 19 01:52:15.505268 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:15.505233    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-45rlh_1e5626e3-1d2c-470c-b619-17ec423c7922/manager/0.log"
Sep 19 01:52:15.992072 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:15.992041    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-cl6v6_74af2f71-487e-406c-b594-3292cde8ea44/manager/0.log"
Sep 19 01:52:16.065557 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:16.065466    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5bc8cb64b6-8cqf4_436955fc-9575-4cb2-bcdc-dec43a78bec6/manager/0.log"
Sep 19 01:52:16.086168 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:16.086140    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-d9vhq_e7192a01-988e-4143-a5bf-37ac7910fcd4/server/0.log"
Sep 19 01:52:16.299549 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:16.299516    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-748475db69-5dq5z_2015c019-405a-4565-9c96-8ddda261051c/manager/0.log"
Sep 19 01:52:16.324917 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:16.324828    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-748475db69-5dq5z_2015c019-405a-4565-9c96-8ddda261051c/copy-manifests/0.log"
Sep 19 01:52:16.958375 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:16.958345    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c6796f956-wzrsb_db018a41-4085-448a-bcae-81fe88b9f53a/kube-auth-proxy/0.log"
Sep 19 01:52:17.013674 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:17.013640    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-bb6qh_ee7c3e3f-b988-4467-82b7-6bda62a1140d/payload-pre-processing/0.log"
Sep 19 01:52:17.054538 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:17.054473    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-42wkt_c9ba5520-238c-4d7b-9266-16a1c0f76e8d/payload-processing/0.log"
Sep 19 01:52:17.082904 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:17.082874    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69d68c9d66-2gblx_9cb55096-dad6-4d87-a63b-48e18dc7a011/router/0.log"
Sep 19 01:52:17.961523 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:17.961423    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-j9kn6_3d3d81d4-e171-4c3b-a1e5-135280d40915/openshift-lws-operator/0.log"
Sep 19 01:52:19.483895 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:19.483862    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c6796f956-wzrsb_db018a41-4085-448a-bcae-81fe88b9f53a/kube-auth-proxy/0.log"
Sep 19 01:52:19.549843 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:19.549787    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-bb6qh_ee7c3e3f-b988-4467-82b7-6bda62a1140d/payload-pre-processing/0.log"
Sep 19 01:52:19.601786 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:19.601749    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-42wkt_c9ba5520-238c-4d7b-9266-16a1c0f76e8d/payload-processing/0.log"
Sep 19 01:52:19.621294 ip-10-0-131-76 kubenswrapper[2580]: I0919 01:52:19.621255    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69d68c9d66-2gblx_9cb55096-dad6-4d87-a63b-48e18dc7a011/router/0.log"