Jul 30 00:43:31.608799 ip-10-0-141-85 systemd[1]: Starting Kubernetes Kubelet...
Jul 30 00:43:32.013096 ip-10-0-141-85 kubenswrapper[2546]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 00:43:32.013096 ip-10-0-141-85 kubenswrapper[2546]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 30 00:43:32.013096 ip-10-0-141-85 kubenswrapper[2546]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 00:43:32.013096 ip-10-0-141-85 kubenswrapper[2546]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 30 00:43:32.013096 ip-10-0-141-85 kubenswrapper[2546]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 00:43:32.015220 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.015125    2546 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 30 00:43:32.023325 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023299    2546 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 00:43:32.023325 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023318    2546 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 00:43:32.023325 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023323    2546 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 00:43:32.023325 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023326    2546 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 00:43:32.023325 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023329    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 00:43:32.023325 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023332    2546 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 00:43:32.023551 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023335    2546 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 00:43:32.023551 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023338    2546 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 00:43:32.023551 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023341    2546 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 00:43:32.023551 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023345    2546 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 00:43:32.023551 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023350    2546 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 00:43:32.023551 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023353    2546 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 00:43:32.023551 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023356    2546 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 00:43:32.023551 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023359    2546 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 00:43:32.023551 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023362    2546 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 00:43:32.023551 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023365    2546 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 00:43:32.023551 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023368    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 00:43:32.023551 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023371    2546 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 00:43:32.023551 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023374    2546 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 00:43:32.023551 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023376    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 00:43:32.023551 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023381    2546 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 00:43:32.023551 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023384    2546 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 00:43:32.023551 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023390    2546 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 00:43:32.023551 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023393    2546 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 00:43:32.023551 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023396    2546 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 00:43:32.023551 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023399    2546 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 00:43:32.024047 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023401    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 00:43:32.024047 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023404    2546 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 00:43:32.024047 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023407    2546 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 00:43:32.024047 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023409    2546 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 00:43:32.024047 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023412    2546 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 00:43:32.024047 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023414    2546 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 00:43:32.024047 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023417    2546 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 00:43:32.024047 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023420    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 00:43:32.024047 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023423    2546 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 00:43:32.024047 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023425    2546 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 00:43:32.024047 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023428    2546 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 00:43:32.024047 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023430    2546 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 00:43:32.024047 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023435    2546 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 00:43:32.024047 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023440    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 00:43:32.024047 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023443    2546 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 00:43:32.024047 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023446    2546 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 00:43:32.024047 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023449    2546 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 00:43:32.024047 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023452    2546 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 00:43:32.024047 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023455    2546 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 00:43:32.024513 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023458    2546 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 00:43:32.024513 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023461    2546 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 00:43:32.024513 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023464    2546 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 00:43:32.024513 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023466    2546 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 00:43:32.024513 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023469    2546 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 00:43:32.024513 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023472    2546 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 00:43:32.024513 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023476    2546 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 00:43:32.024513 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023479    2546 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 00:43:32.024513 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023481    2546 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 00:43:32.024513 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023484    2546 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 00:43:32.024513 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023487    2546 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 00:43:32.024513 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023490    2546 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 00:43:32.024513 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023506    2546 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 00:43:32.024513 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023509    2546 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 00:43:32.024513 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023512    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 00:43:32.024513 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023515    2546 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 00:43:32.024513 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023518    2546 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 00:43:32.024513 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023520    2546 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 00:43:32.024513 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023523    2546 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 00:43:32.024513 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023525    2546 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 00:43:32.025042 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023529    2546 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 00:43:32.025042 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023531    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 00:43:32.025042 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023534    2546 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 00:43:32.025042 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023537    2546 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 00:43:32.025042 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023539    2546 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 00:43:32.025042 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023542    2546 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 00:43:32.025042 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023545    2546 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 00:43:32.025042 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023548    2546 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 00:43:32.025042 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023551    2546 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 00:43:32.025042 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023554    2546 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 00:43:32.025042 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023557    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 00:43:32.025042 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023559    2546 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 00:43:32.025042 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023562    2546 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 00:43:32.025042 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023565    2546 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 00:43:32.025042 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023567    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 00:43:32.025042 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023570    2546 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 00:43:32.025042 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023572    2546 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 00:43:32.025042 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023575    2546 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 00:43:32.025042 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023577    2546 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 00:43:32.025042 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023580    2546 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 00:43:32.025547 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023583    2546 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 00:43:32.025547 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023585    2546 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 00:43:32.025547 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023587    2546 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 00:43:32.025547 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023592    2546 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 00:43:32.025547 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023594    2546 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 00:43:32.025547 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023597    2546 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 00:43:32.025547 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023600    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 00:43:32.025547 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023603    2546 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 00:43:32.025547 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023605    2546 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 00:43:32.025547 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023608    2546 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 00:43:32.025547 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023610    2546 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 00:43:32.025547 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023613    2546 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 00:43:32.025547 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023617    2546 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 00:43:32.025547 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.023620    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 00:43:32.025547 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024379    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 00:43:32.025547 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024386    2546 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 00:43:32.025547 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024390    2546 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 00:43:32.025547 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024393    2546 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 00:43:32.025547 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024396    2546 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 00:43:32.026003 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024399    2546 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 00:43:32.026003 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024402    2546 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 00:43:32.026003 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024405    2546 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 00:43:32.026003 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024407    2546 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 00:43:32.026003 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024410    2546 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 00:43:32.026003 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024413    2546 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 00:43:32.026003 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024415    2546 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 00:43:32.026003 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024418    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 00:43:32.026003 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024421    2546 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 00:43:32.026003 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024423    2546 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 00:43:32.026003 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024426    2546 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 00:43:32.026003 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024429    2546 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 00:43:32.026003 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024431    2546 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 00:43:32.026003 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024434    2546 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 00:43:32.026003 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024437    2546 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 00:43:32.026003 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024439    2546 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 00:43:32.026003 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024443    2546 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 00:43:32.026003 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024448    2546 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 00:43:32.026003 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024450    2546 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 00:43:32.026003 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024453    2546 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 00:43:32.026489 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024455    2546 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 00:43:32.026489 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024458    2546 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 00:43:32.026489 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024461    2546 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 00:43:32.026489 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024463    2546 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 00:43:32.026489 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024466    2546 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 00:43:32.026489 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024469    2546 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 00:43:32.026489 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024472    2546 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 00:43:32.026489 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024475    2546 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 00:43:32.026489 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024479    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 00:43:32.026489 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024482    2546 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 00:43:32.026489 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024485    2546 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 00:43:32.026489 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024488    2546 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 00:43:32.026489 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024490    2546 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 00:43:32.026489 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024509    2546 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 00:43:32.026489 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024512    2546 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 00:43:32.026489 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024514    2546 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 00:43:32.026489 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024518    2546 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 00:43:32.026489 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024521    2546 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 00:43:32.026489 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024524    2546 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 00:43:32.026973 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024526    2546 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 00:43:32.026973 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024529    2546 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 00:43:32.026973 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024532    2546 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 00:43:32.026973 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024535    2546 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 00:43:32.026973 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024537    2546 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 00:43:32.026973 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024540    2546 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 00:43:32.026973 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024542    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 00:43:32.026973 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024545    2546 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 00:43:32.026973 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024548    2546 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 00:43:32.026973 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024551    2546 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 00:43:32.026973 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024554    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 00:43:32.026973 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024557    2546 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 00:43:32.026973 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024560    2546 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 00:43:32.026973 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024563    2546 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 00:43:32.026973 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024565    2546 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 00:43:32.026973 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024568    2546 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 00:43:32.026973 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024571    2546 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 00:43:32.026973 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024573    2546 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 00:43:32.026973 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024576    2546 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 00:43:32.026973 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024579    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 00:43:32.027451 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024581    2546 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 00:43:32.027451 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024584    2546 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 00:43:32.027451 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024586    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 00:43:32.027451 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024589    2546 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 00:43:32.027451 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024592    2546 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 00:43:32.027451 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024594    2546 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 00:43:32.027451 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024597    2546 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 00:43:32.027451 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024600    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 00:43:32.027451 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024603    2546 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 00:43:32.027451 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024605    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 00:43:32.027451 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024608    2546 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 00:43:32.027451 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024610    2546 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 00:43:32.027451 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024613    2546 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 00:43:32.027451 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024615    2546 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 00:43:32.027451 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024618    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 00:43:32.027451 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024622    2546 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 00:43:32.027451 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024625    2546 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 00:43:32.027451 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024629    2546 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 00:43:32.027451 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024633    2546 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 00:43:32.027922 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024636    2546 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 00:43:32.027922 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024638    2546 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 00:43:32.027922 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024641    2546 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 00:43:32.027922 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024645    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 00:43:32.027922 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024647    2546 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 00:43:32.027922 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024650    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 00:43:32.027922 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024653    2546 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 00:43:32.027922 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024656    2546 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 00:43:32.027922 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024660    2546 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 00:43:32.027922 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024663    2546 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 00:43:32.027922 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024666    2546 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 00:43:32.027922 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024669    2546 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 00:43:32.027922 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024672    2546 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 00:43:32.027922 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024675    2546 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 00:43:32.027922 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024677    2546 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 00:43:32.027922 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.024680    2546 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 00:43:32.027922 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025067    2546 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 00:43:32.027922 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025072    2546 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 00:43:32.027922 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025076    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 00:43:32.028380 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025079    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 00:43:32.028380 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025081    2546 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 00:43:32.028380 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025084    2546 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 00:43:32.028380 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025087    2546 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 00:43:32.028380 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025090    2546 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 00:43:32.028380 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025094    2546 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 00:43:32.028380 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025097    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 00:43:32.028380 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025099    2546 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 00:43:32.028380 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025102    2546 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 00:43:32.028380 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025105    2546 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 00:43:32.028380 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025107    2546 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 00:43:32.028380 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025110    2546 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 00:43:32.028380 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025113    2546 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 00:43:32.028380 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025115    2546 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 00:43:32.028380 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025118    2546 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 00:43:32.028380 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025121    2546 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 00:43:32.028380 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025124    2546 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 00:43:32.028380 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025126    2546 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 00:43:32.028380 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025130    2546 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 00:43:32.028861 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025132    2546 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 00:43:32.028861 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025135    2546 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 00:43:32.028861 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025137    2546 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 00:43:32.028861 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025140    2546 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 00:43:32.028861 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025142    2546 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 00:43:32.028861 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025145    2546 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 00:43:32.028861 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025147    2546 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 00:43:32.028861 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025149    2546 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 00:43:32.028861 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025152    2546 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 00:43:32.028861 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025154    2546 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 00:43:32.028861 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025157    2546 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 00:43:32.028861 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025159    2546 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 00:43:32.028861 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025161    2546 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 00:43:32.028861 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025165    2546 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 00:43:32.028861 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025167    2546 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 00:43:32.028861 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025170    2546 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 00:43:32.028861 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025172    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 00:43:32.028861 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025175    2546 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 00:43:32.028861 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025178    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 00:43:32.029348 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025180    2546 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 00:43:32.029348 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025183    2546 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 00:43:32.029348 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025185    2546 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 00:43:32.029348 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025188    2546 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 00:43:32.029348 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025190    2546 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 00:43:32.029348 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025193    2546 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 00:43:32.029348 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025195    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 00:43:32.029348 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025198    2546 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 00:43:32.029348 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025200    2546 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 00:43:32.029348 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025204    2546 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 00:43:32.029348 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025206    2546 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 00:43:32.029348 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025209    2546 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 00:43:32.029348 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025211    2546 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 00:43:32.029348 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025214    2546 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 00:43:32.029348 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025217    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 00:43:32.029348 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025219    2546 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 00:43:32.029348 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025222    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 00:43:32.029348 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025224    2546 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 00:43:32.029348 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025227    2546 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 00:43:32.029348 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025230    2546 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 00:43:32.029944 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025232    2546 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 00:43:32.029944 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025234    2546 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 00:43:32.029944 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025237    2546 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 00:43:32.029944 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025239    2546 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 00:43:32.029944 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025242    2546 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 00:43:32.029944 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025245    2546 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 00:43:32.029944 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025247    2546 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 00:43:32.029944 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025250    2546 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 00:43:32.029944 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025253    2546 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 00:43:32.029944 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025255    2546 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 00:43:32.029944 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025258    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 00:43:32.029944 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025262    2546 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 00:43:32.029944 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025264    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 00:43:32.029944 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025267    2546 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 00:43:32.029944 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025269    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 00:43:32.029944 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025272    2546 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 00:43:32.029944 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025274    2546 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 00:43:32.029944 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025277    2546 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 00:43:32.029944 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025279    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 00:43:32.029944 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025282    2546 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 00:43:32.030456 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025284    2546 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 00:43:32.030456 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025290    2546 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 00:43:32.030456 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025292    2546 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 00:43:32.030456 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025295    2546 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 00:43:32.030456 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025298    2546 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 00:43:32.030456 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025302    2546 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 00:43:32.030456 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025306    2546 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 00:43:32.030456 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025308    2546 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 00:43:32.030456 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025311    2546 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 00:43:32.030456 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025314    2546 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 00:43:32.030456 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025316    2546 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 00:43:32.030456 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025319    2546 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 00:43:32.030456 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025322    2546 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 00:43:32.030456 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025324    2546 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 00:43:32.030456 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025327    2546 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 00:43:32.030456 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025329    2546 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 00:43:32.030456 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025332    2546 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 00:43:32.030456 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.025335    2546 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 00:43:32.030456 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025410    2546 flags.go:64] FLAG: --address="0.0.0.0"
Jul 30 00:43:32.030456 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025418    2546 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 30 00:43:32.030456 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025424    2546 flags.go:64] FLAG: --anonymous-auth="true"
Jul 30 00:43:32.030977 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025429    2546 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 30 00:43:32.030977 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025433    2546 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 30 00:43:32.030977 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025436    2546 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 30 00:43:32.030977 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025441    2546 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 30 00:43:32.030977 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025445    2546 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 30 00:43:32.030977 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025449    2546 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 30 00:43:32.030977 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025452    2546 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 30 00:43:32.030977 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025456    2546 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 30 00:43:32.030977 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025459    2546 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 30 00:43:32.030977 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025462    2546 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 30 00:43:32.030977 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025465    2546 flags.go:64] FLAG: --cgroup-root=""
Jul 30 00:43:32.030977 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025468    2546 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 30 00:43:32.030977 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025473    2546 flags.go:64] FLAG: --client-ca-file=""
Jul 30 00:43:32.030977 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025477    2546 flags.go:64] FLAG: --cloud-provider="external"
Jul 30 00:43:32.030977 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025480    2546 flags.go:64] FLAG: --cluster-dns="[]"
Jul 30 00:43:32.030977 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025484    2546 flags.go:64] FLAG: --cluster-domain=""
Jul 30 00:43:32.030977 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025488    2546 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 30 00:43:32.030977 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025491    2546 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 30 00:43:32.030977 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025508    2546 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 30 00:43:32.030977 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025512    2546 flags.go:64] FLAG: --container-log-max-files="5"
Jul 30 00:43:32.030977 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025516    2546 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 30 00:43:32.030977 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025519    2546 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 30 00:43:32.030977 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025522    2546 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025525    2546 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025528    2546 flags.go:64] FLAG: --contention-profiling="false"
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025531    2546 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025534    2546 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025538    2546 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025541    2546 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025545    2546 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025548    2546 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025551    2546 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025555    2546 flags.go:64] FLAG: --enable-load-reader="false"
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025558    2546 flags.go:64] FLAG: --enable-server="true"
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025561    2546 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025565    2546 flags.go:64] FLAG: --event-burst="100"
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025569    2546 flags.go:64] FLAG: --event-qps="50"
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025571    2546 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025575    2546 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025578    2546 flags.go:64] FLAG: --eviction-hard=""
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025581    2546 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025584    2546 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025587    2546 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025591    2546 flags.go:64] FLAG: --eviction-soft=""
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025594    2546 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025597    2546 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025600    2546 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 30 00:43:32.031589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025603    2546 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 30 00:43:32.032223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025606    2546 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 30 00:43:32.032223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025609    2546 flags.go:64] FLAG: --fail-swap-on="true"
Jul 30 00:43:32.032223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025612    2546 flags.go:64] FLAG: --feature-gates=""
Jul 30 00:43:32.032223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025616    2546 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 30 00:43:32.032223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025619    2546 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 30 00:43:32.032223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025622    2546 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 30 00:43:32.032223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025625    2546 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 30 00:43:32.032223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025630    2546 flags.go:64] FLAG: --healthz-port="10248"
Jul 30 00:43:32.032223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025633    2546 flags.go:64] FLAG: --help="false"
Jul 30 00:43:32.032223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025636    2546 flags.go:64] FLAG: --hostname-override="ip-10-0-141-85.ec2.internal"
Jul 30 00:43:32.032223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025640    2546 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 30 00:43:32.032223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025642    2546 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 30 00:43:32.032223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025646    2546 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 30 00:43:32.032223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025649    2546 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 30 00:43:32.032223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025653    2546 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 30 00:43:32.032223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025656    2546 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 30 00:43:32.032223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025659    2546 flags.go:64] FLAG: --image-service-endpoint=""
Jul 30 00:43:32.032223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025662    2546 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 30 00:43:32.032223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025665    2546 flags.go:64] FLAG: --kube-api-burst="100"
Jul 30 00:43:32.032223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025668    2546 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 30 00:43:32.032223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025672    2546 flags.go:64] FLAG: --kube-api-qps="50"
Jul 30 00:43:32.032223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025675    2546 flags.go:64] FLAG: --kube-reserved=""
Jul 30 00:43:32.032223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025678    2546 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 30 00:43:32.032223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025680    2546 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025683    2546 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025686    2546 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025689    2546 flags.go:64] FLAG: --lock-file=""
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025692    2546 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025695    2546 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025698    2546 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025704    2546 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025707    2546 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025710    2546 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025713    2546 flags.go:64] FLAG: --logging-format="text"
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025716    2546 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025719    2546 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025722    2546 flags.go:64] FLAG: --manifest-url=""
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025725    2546 flags.go:64] FLAG: --manifest-url-header=""
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025730    2546 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025735    2546 flags.go:64] FLAG: --max-open-files="1000000"
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025739    2546 flags.go:64] FLAG: --max-pods="110"
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025742    2546 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025745    2546 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025748    2546 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025751    2546 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025754    2546 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025757    2546 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025760    2546 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 30 00:43:32.032838 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025772    2546 flags.go:64] FLAG: --node-status-max-images="50"
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025775    2546 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025778    2546 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025781    2546 flags.go:64] FLAG: --pod-cidr=""
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025784    2546 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025790    2546 flags.go:64] FLAG: --pod-manifest-path=""
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025793    2546 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025796    2546 flags.go:64] FLAG: --pods-per-core="0"
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025799    2546 flags.go:64] FLAG: --port="10250"
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025802    2546 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025805    2546 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0bca0cd7f74248e78"
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025808    2546 flags.go:64] FLAG: --qos-reserved=""
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025811    2546 flags.go:64] FLAG: --read-only-port="10255"
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025814    2546 flags.go:64] FLAG: --register-node="true"
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025817    2546 flags.go:64] FLAG: --register-with-taints=""
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025820    2546 flags.go:64] FLAG: --registry-burst="10"
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025823    2546 flags.go:64] FLAG: --registry-qps="5"
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025826    2546 flags.go:64] FLAG: --reserved-cpus=""
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025829    2546 flags.go:64] FLAG: --reserved-memory=""
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025833    2546 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025836    2546 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025839    2546 flags.go:64] FLAG: --rotate-certificates="false"
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025842    2546 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025846    2546 flags.go:64] FLAG: --runonce="false"
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025849    2546 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 30 00:43:32.033726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025853    2546 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025856    2546 flags.go:64] FLAG: --seccomp-default="false"
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025858    2546 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025861    2546 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025864    2546 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025868    2546 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025871    2546 flags.go:64] FLAG: --storage-driver-password="root"
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025873    2546 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025876    2546 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025879    2546 flags.go:64] FLAG: --storage-driver-user="root"
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025882    2546 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025886    2546 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025889    2546 flags.go:64] FLAG: --system-cgroups=""
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025891    2546 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025897    2546 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025900    2546 flags.go:64] FLAG: --tls-cert-file=""
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025902    2546 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025907    2546 flags.go:64] FLAG: --tls-min-version=""
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025909    2546 flags.go:64] FLAG: --tls-private-key-file=""
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025912    2546 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025915    2546 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025918    2546 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025921    2546 flags.go:64] FLAG: --v="2"
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025925    2546 flags.go:64] FLAG: --version="false"
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025929    2546 flags.go:64] FLAG: --vmodule=""
Jul 30 00:43:32.034516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025933    2546 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 30 00:43:32.035140 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.025936    2546 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 30 00:43:32.035140 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026048    2546 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 00:43:32.035140 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026051    2546 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 00:43:32.035140 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026055    2546 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 00:43:32.035140 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026060    2546 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 00:43:32.035140 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026062    2546 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 00:43:32.035140 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026065    2546 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 00:43:32.035140 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026067    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 00:43:32.035140 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026070    2546 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 00:43:32.035140 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026072    2546 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 00:43:32.035140 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026075    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 00:43:32.035140 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026079    2546 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 00:43:32.035140 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026082    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 00:43:32.035140 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026084    2546 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 00:43:32.035140 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026087    2546 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 00:43:32.035140 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026090    2546 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 00:43:32.035140 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026092    2546 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 00:43:32.035140 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026095    2546 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 00:43:32.035140 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026098    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 00:43:32.035140 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026100    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 00:43:32.035740 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026103    2546 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 00:43:32.035740 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026106    2546 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 00:43:32.035740 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026108    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 00:43:32.035740 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026111    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 00:43:32.035740 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026114    2546 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 00:43:32.035740 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026117    2546 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 00:43:32.035740 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026120    2546 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 00:43:32.035740 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026123    2546 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 00:43:32.035740 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026125    2546 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 00:43:32.035740 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026128    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 00:43:32.035740 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026130    2546 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 00:43:32.035740 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026133    2546 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 00:43:32.035740 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026135    2546 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 00:43:32.035740 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026138    2546 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 00:43:32.035740 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026141    2546 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 00:43:32.035740 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026145    2546 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 00:43:32.035740 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026151    2546 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 00:43:32.035740 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026153    2546 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 00:43:32.035740 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026156    2546 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 00:43:32.035740 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026158    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 00:43:32.036215 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026161    2546 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 00:43:32.036215 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026163    2546 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 00:43:32.036215 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026166    2546 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 00:43:32.036215 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026169    2546 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 00:43:32.036215 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026173    2546 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 00:43:32.036215 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026175    2546 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 00:43:32.036215 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026178    2546 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 00:43:32.036215 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026180    2546 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 00:43:32.036215 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026183    2546 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 00:43:32.036215 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026185    2546 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 00:43:32.036215 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026188    2546 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 00:43:32.036215 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026191    2546 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 00:43:32.036215 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026193    2546 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 00:43:32.036215 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026196    2546 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 00:43:32.036215 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026198    2546 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 00:43:32.036215 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026201    2546 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 00:43:32.036215 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026204    2546 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 00:43:32.036215 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026206    2546 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 00:43:32.036215 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026209    2546 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 00:43:32.036694 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026212    2546 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 00:43:32.036694 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026215    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 00:43:32.036694 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026219    2546 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 00:43:32.036694 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026221    2546 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 00:43:32.036694 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026224    2546 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 00:43:32.036694 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026227    2546 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 00:43:32.036694 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026229    2546 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 00:43:32.036694 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026232    2546 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 00:43:32.036694 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026235    2546 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 00:43:32.036694 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026238    2546 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 00:43:32.036694 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026241    2546 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 00:43:32.036694 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026244    2546 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 00:43:32.036694 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026246    2546 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 00:43:32.036694 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026249    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 00:43:32.036694 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026251    2546 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 00:43:32.036694 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026254    2546 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 00:43:32.036694 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026258    2546 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 00:43:32.036694 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026261    2546 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 00:43:32.036694 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026263    2546 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 00:43:32.037148 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026266    2546 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 00:43:32.037148 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026269    2546 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 00:43:32.037148 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026271    2546 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 00:43:32.037148 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026274    2546 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 00:43:32.037148 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026276    2546 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 00:43:32.037148 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026279    2546 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 00:43:32.037148 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026282    2546 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 00:43:32.037148 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026284    2546 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 00:43:32.037148 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026287    2546 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 00:43:32.037148 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026290    2546 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 00:43:32.037148 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026292    2546 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 00:43:32.037148 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026295    2546 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 00:43:32.037148 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026297    2546 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 00:43:32.037148 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026300    2546 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 00:43:32.037148 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026303    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 00:43:32.037148 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026305    2546 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 00:43:32.037148 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026308    2546 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 00:43:32.037148 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026310    2546 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 00:43:32.037148 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026313    2546 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 00:43:32.037148 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026315    2546 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 00:43:32.037651 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026318    2546 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 00:43:32.037651 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.026322    2546 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 00:43:32.037651 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.027041    2546 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 00:43:32.037651 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.035038    2546 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 30 00:43:32.037651 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.035057    2546 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 30 00:43:32.037651 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.035077    2546 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 30 00:43:32.037651 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.035083    2546 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 30 00:43:32.037651 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035141    2546 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 00:43:32.037651 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035146    2546 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 00:43:32.037651 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035150    2546 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 00:43:32.037651 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035154    2546 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 00:43:32.037651 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035157    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 00:43:32.037651 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035160    2546 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 00:43:32.037651 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035162    2546 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 00:43:32.038013 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035165    2546 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 00:43:32.038013 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035168    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 00:43:32.038013 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035171    2546 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 00:43:32.038013 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035176    2546 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 00:43:32.038013 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035179    2546 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 00:43:32.038013 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035184    2546 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 00:43:32.038013 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035188    2546 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 00:43:32.038013 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035191    2546 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 00:43:32.038013 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035194    2546 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 00:43:32.038013 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035197    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 00:43:32.038013 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035201    2546 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 00:43:32.038013 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035205    2546 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 00:43:32.038013 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035208    2546 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 00:43:32.038013 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035211    2546 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 00:43:32.038013 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035213    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 00:43:32.038013 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035216    2546 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 00:43:32.038013 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035218    2546 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 00:43:32.038013 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035221    2546 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 00:43:32.038013 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035224    2546 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 00:43:32.038013 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035226    2546 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 00:43:32.038523 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035229    2546 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 00:43:32.038523 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035232    2546 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 00:43:32.038523 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035235    2546 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 00:43:32.038523 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035238    2546 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 00:43:32.038523 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035241    2546 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 00:43:32.038523 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035244    2546 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 00:43:32.038523 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035246    2546 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 00:43:32.038523 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035249    2546 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 00:43:32.038523 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035251    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 00:43:32.038523 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035254    2546 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 00:43:32.038523 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035257    2546 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 00:43:32.038523 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035259    2546 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 00:43:32.038523 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035262    2546 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 00:43:32.038523 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035264    2546 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 00:43:32.038523 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035267    2546 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 00:43:32.038523 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035269    2546 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 00:43:32.038523 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035272    2546 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 00:43:32.038523 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035274    2546 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 00:43:32.038523 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035277    2546 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 00:43:32.039016 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035280    2546 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 00:43:32.039016 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035283    2546 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 00:43:32.039016 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035285    2546 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 00:43:32.039016 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035287    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 00:43:32.039016 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035290    2546 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 00:43:32.039016 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035292    2546 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 00:43:32.039016 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035295    2546 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 00:43:32.039016 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035298    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 00:43:32.039016 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035301    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 00:43:32.039016 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035303    2546 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 00:43:32.039016 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035306    2546 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 00:43:32.039016 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035308    2546 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 00:43:32.039016 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035311    2546 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 00:43:32.039016 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035313    2546 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 00:43:32.039016 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035316    2546 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 00:43:32.039016 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035319    2546 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 00:43:32.039016 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035322    2546 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 00:43:32.039016 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035324    2546 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 00:43:32.039016 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035328    2546 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 00:43:32.039016 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035332    2546 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 00:43:32.039615 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035334    2546 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 00:43:32.039615 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035337    2546 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 00:43:32.039615 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035340    2546 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 00:43:32.039615 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035342    2546 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 00:43:32.039615 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035345    2546 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 00:43:32.039615 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035348    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 00:43:32.039615 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035351    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 00:43:32.039615 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035354    2546 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 00:43:32.039615 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035356    2546 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 00:43:32.039615 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035359    2546 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 00:43:32.039615 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035361    2546 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 00:43:32.039615 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035363    2546 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 00:43:32.039615 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035366    2546 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 00:43:32.039615 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035370    2546 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 00:43:32.039615 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035372    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 00:43:32.039615 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035375    2546 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 00:43:32.039615 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035377    2546 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 00:43:32.039615 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035380    2546 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 00:43:32.039615 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035382    2546 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 00:43:32.040077 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035385    2546 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 00:43:32.040077 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035387    2546 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 00:43:32.040077 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035390    2546 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 00:43:32.040077 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035392    2546 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 00:43:32.040077 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035395    2546 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 00:43:32.040077 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035398    2546 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 00:43:32.040077 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035400    2546 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 00:43:32.040077 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035402    2546 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 00:43:32.040077 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035406    2546 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 00:43:32.040077 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035409    2546 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 00:43:32.040077 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035412    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 00:43:32.040077 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035414    2546 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 00:43:32.040077 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035417    2546 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 00:43:32.040077 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035419    2546 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 00:43:32.040077 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.035424    2546 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 00:43:32.040457 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035556    2546 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 00:43:32.040457 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035561    2546 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 00:43:32.040457 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035566    2546 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 00:43:32.040457 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035569    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 00:43:32.040457 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035572    2546 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 00:43:32.040457 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035576    2546 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 00:43:32.040457 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035580    2546 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 00:43:32.040457 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035583    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 00:43:32.040457 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035585    2546 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 00:43:32.040457 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035588    2546 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 00:43:32.040457 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035591    2546 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 00:43:32.040457 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035595    2546 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 00:43:32.040457 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035600    2546 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 00:43:32.040457 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035603    2546 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 00:43:32.040457 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035606    2546 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 00:43:32.040457 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035609    2546 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 00:43:32.040457 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035611    2546 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 00:43:32.040457 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035614    2546 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 00:43:32.040457 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035616    2546 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 00:43:32.040941 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035619    2546 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 00:43:32.040941 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035621    2546 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 00:43:32.040941 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035624    2546 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 00:43:32.040941 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035626    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 00:43:32.040941 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035629    2546 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 00:43:32.040941 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035631    2546 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 00:43:32.040941 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035635    2546 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 00:43:32.040941 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035638    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 00:43:32.040941 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035640    2546 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 00:43:32.040941 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035643    2546 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 00:43:32.040941 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035646    2546 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 00:43:32.040941 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035648    2546 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 00:43:32.040941 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035651    2546 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 00:43:32.040941 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035653    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 00:43:32.040941 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035656    2546 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 00:43:32.040941 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035658    2546 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 00:43:32.040941 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035661    2546 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 00:43:32.040941 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035663    2546 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 00:43:32.040941 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035666    2546 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 00:43:32.040941 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035669    2546 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 00:43:32.041428 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035671    2546 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 00:43:32.041428 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035674    2546 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 00:43:32.041428 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035677    2546 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 00:43:32.041428 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035679    2546 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 00:43:32.041428 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035682    2546 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 00:43:32.041428 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035685    2546 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 00:43:32.041428 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035688    2546 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 00:43:32.041428 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035690    2546 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 00:43:32.041428 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035693    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 00:43:32.041428 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035695    2546 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 00:43:32.041428 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035698    2546 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 00:43:32.041428 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035700    2546 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 00:43:32.041428 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035703    2546 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 00:43:32.041428 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035706    2546 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 00:43:32.041428 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035709    2546 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 00:43:32.041428 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035711    2546 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 00:43:32.041428 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035714    2546 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 00:43:32.041428 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035716    2546 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 00:43:32.041428 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035719    2546 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 00:43:32.041428 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035722    2546 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 00:43:32.041928 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035725    2546 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 00:43:32.041928 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035727    2546 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 00:43:32.041928 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035730    2546 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 00:43:32.041928 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035732    2546 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 00:43:32.041928 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035735    2546 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 00:43:32.041928 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035738    2546 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 00:43:32.041928 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035740    2546 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 00:43:32.041928 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035743    2546 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 00:43:32.041928 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035746    2546 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 00:43:32.041928 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035749    2546 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 00:43:32.041928 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035751    2546 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 00:43:32.041928 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035754    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 00:43:32.041928 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035756    2546 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 00:43:32.041928 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035759    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 00:43:32.041928 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035762    2546 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 00:43:32.041928 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035764    2546 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 00:43:32.041928 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035767    2546 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 00:43:32.041928 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035769    2546 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 00:43:32.041928 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035772    2546 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 00:43:32.041928 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035775    2546 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 00:43:32.042414 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035778    2546 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 00:43:32.042414 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035781    2546 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 00:43:32.042414 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035783    2546 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 00:43:32.042414 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035786    2546 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 00:43:32.042414 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035788    2546 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 00:43:32.042414 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035791    2546 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 00:43:32.042414 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035794    2546 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 00:43:32.042414 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035796    2546 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 00:43:32.042414 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035798    2546 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 00:43:32.042414 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035801    2546 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 00:43:32.042414 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035803    2546 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 00:43:32.042414 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035806    2546 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 00:43:32.042414 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035809    2546 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 00:43:32.042414 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035811    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 00:43:32.042414 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035814    2546 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 00:43:32.042414 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035817    2546 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 00:43:32.042414 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035819    2546 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 00:43:32.042414 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035822    2546 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 00:43:32.042414 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035824    2546 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 00:43:32.042414 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:32.035827    2546 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 00:43:32.042939 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.035832    2546 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 00:43:32.042939 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.035970    2546 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 30 00:43:32.042939 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.038785    2546 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 30 00:43:32.042939 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.040386    2546 server.go:1019] "Starting client certificate rotation"
Jul 30 00:43:32.042939 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.040484    2546 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 00:43:32.042939 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.040532    2546 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 00:43:32.043107 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.042972    2546 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 00:43:32.044898 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.044878    2546 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 00:43:32.054644 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.054627    2546 log.go:25] "Validated CRI v1 runtime API"
Jul 30 00:43:32.058278 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.058258    2546 log.go:25] "Validated CRI v1 image API"
Jul 30 00:43:32.059410 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.059394    2546 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 30 00:43:32.061866 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.061843    2546 fs.go:135] Filesystem UUIDs: map[396de06d-bb69-49ca-b66b-61043bc76737:/dev/nvme0n1p4 654388de-56fe-4f34-9eea-29cc3cc4b6bd:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 30 00:43:32.061935 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.061861    2546 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 30 00:43:32.065915 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.065803    2546 manager.go:216] Machine: {Timestamp:2026-07-30 00:43:32.065325409 +0000 UTC m=+0.336084496 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3110109 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23c489aa954e4cd717497177e32723 SystemUUID:ec23c489-aa95-4e4c-d717-497177e32723 BootID:b4b1de39-ed2b-4490-a971-2f7572774194 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:cd:02:fa:0e:97 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:cd:02:fa:0e:97 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:26:21:80:74:5e:22 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 30 00:43:32.065915 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.065907    2546 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 30 00:43:32.066043 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.065985    2546 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 30 00:43:32.069272 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.069248    2546 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 30 00:43:32.069775 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.069273    2546 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-85.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 30 00:43:32.070570 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.070541    2546 topology_manager.go:138] "Creating topology manager with none policy"
Jul 30 00:43:32.070688 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.070674    2546 container_manager_linux.go:309] "Creating device plugin manager"
Jul 30 00:43:32.070688 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.070689    2546 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 00:43:32.070852 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.070705    2546 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 00:43:32.070852 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.070643    2546 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 00:43:32.071603 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.071589    2546 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 30 00:43:32.073375 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.073363    2546 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 00:43:32.073510 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.073487    2546 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 30 00:43:32.073554 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.073532    2546 kubelet.go:486] "Attempting to sync node with API server"
Jul 30 00:43:32.073554 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.073547    2546 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 30 00:43:32.073606 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.073565    2546 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 30 00:43:32.073606 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.073578    2546 kubelet.go:398] "Adding apiserver pod source"
Jul 30 00:43:32.073606 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.073590    2546 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 30 00:43:32.073999 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.073988    2546 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 00:43:32.074041 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.074007    2546 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 00:43:32.076254 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.076232    2546 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 30 00:43:32.078097 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.078084    2546 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 30 00:43:32.078137 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.078114    2546 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 30 00:43:32.079804 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.079790    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 30 00:43:32.079804 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.079805    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 30 00:43:32.079882 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.079811    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 30 00:43:32.079882 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.079818    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 30 00:43:32.079882 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.079826    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 30 00:43:32.079882 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.079832    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 30 00:43:32.079882 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.079840    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 30 00:43:32.079882 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.079846    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 30 00:43:32.079882 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.079853    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 30 00:43:32.079882 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.079858    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 30 00:43:32.079882 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.079867    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 30 00:43:32.079882 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.079872    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 30 00:43:32.081377 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.081368    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 30 00:43:32.081377 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.081377    2546 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 30 00:43:32.082960 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.082944    2546 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-85.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 30 00:43:32.083516 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:32.083482    2546 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 30 00:43:32.083577 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:32.083481    2546 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 30 00:43:32.084464 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.084452    2546 server.go:1268] "Started kubelet"
Jul 30 00:43:32.084633 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.084582    2546 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 30 00:43:32.084736 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.084622    2546 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 30 00:43:32.084736 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.084655    2546 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 30 00:43:32.085337 ip-10-0-141-85 systemd[1]: Started Kubernetes Kubelet.
Jul 30 00:43:32.086285 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.086221    2546 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 30 00:43:32.088591 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.088576    2546 server.go:310] "Adding debug handlers to kubelet server"
Jul 30 00:43:32.090914 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.090896    2546 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4lv6k"
Jul 30 00:43:32.091146 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:32.091131    2546 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 30 00:43:32.091814 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:32.090808    2546 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-85.ec2.internal.18c6ea914d2541cd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-85.ec2.internal,UID:ip-10-0-141-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-85.ec2.internal,},FirstTimestamp:2026-07-30 00:43:32.084425165 +0000 UTC m=+0.355184251,LastTimestamp:2026-07-30 00:43:32.084425165 +0000 UTC m=+0.355184251,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-85.ec2.internal,}"
Jul 30 00:43:32.093575 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.093551    2546 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 30 00:43:32.093575 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.093556    2546 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 30 00:43:32.093712 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:32.093669    2546 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-85.ec2.internal\" not found"
Jul 30 00:43:32.094320 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.094218    2546 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 30 00:43:32.094407 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.094335    2546 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 30 00:43:32.094407 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.094351    2546 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 30 00:43:32.094525 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.094514    2546 reconstruct.go:97] "Volume reconstruction finished"
Jul 30 00:43:32.094525 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.094522    2546 reconciler.go:29] "Reconciler: start to sync state"
Jul 30 00:43:32.094721 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.094704    2546 factory.go:153] Registering CRI-O factory
Jul 30 00:43:32.094792 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.094725    2546 factory.go:223] Registration of the crio container factory successfully
Jul 30 00:43:32.094868 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.094822    2546 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 30 00:43:32.094868 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.094846    2546 factory.go:55] Registering systemd factory
Jul 30 00:43:32.094868 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.094855    2546 factory.go:223] Registration of the systemd container factory successfully
Jul 30 00:43:32.095233 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.094873    2546 factory.go:103] Registering Raw factory
Jul 30 00:43:32.095233 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.094883    2546 manager.go:1196] Started watching for new ooms in manager
Jul 30 00:43:32.095476 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.095461    2546 manager.go:349] Starting recovery of all containers
Jul 30 00:43:32.101597 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.101565    2546 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4lv6k"
Jul 30 00:43:32.105649 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:32.105578    2546 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 30 00:43:32.105733 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:32.105671    2546 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 30 00:43:32.109314 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.109293    2546 manager.go:354] Recovery completed
Jul 30 00:43:32.113595 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.113583    2546 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 00:43:32.115537 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.115522    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 00:43:32.115604 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.115551    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 00:43:32.115604 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.115563    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientPID"
Jul 30 00:43:32.116068 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.116053    2546 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 30 00:43:32.116068 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.116067    2546 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 30 00:43:32.116068 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.116073    2546 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 00:43:32.116225 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.116085    2546 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 00:43:32.116743 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.116729    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 00:43:32.116797 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.116755    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 00:43:32.116797 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.116764    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientPID"
Jul 30 00:43:32.117375 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:32.117279    2546 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-85.ec2.internal.18c6ea914efffea0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-85.ec2.internal,UID:ip-10-0-141-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-85.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-85.ec2.internal,},FirstTimestamp:2026-07-30 00:43:32.115537568 +0000 UTC m=+0.386296654,LastTimestamp:2026-07-30 00:43:32.115537568 +0000 UTC m=+0.386296654,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-85.ec2.internal,}"
Jul 30 00:43:32.117792 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.117779    2546 policy_none.go:49] "None policy: Start"
Jul 30 00:43:32.117906 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.117797    2546 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 30 00:43:32.117906 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.117808    2546 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 30 00:43:32.119126 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.119115    2546 policy_none.go:47] "Start"
Jul 30 00:43:32.161133 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.161116    2546 manager.go:337] "Starting Device Plugin manager"
Jul 30 00:43:32.161245 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:32.161153    2546 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 30 00:43:32.161245 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.161163    2546 server.go:87] "Starting device plugin registration server"
Jul 30 00:43:32.161481 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.161467    2546 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 30 00:43:32.161547 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.161485    2546 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 30 00:43:32.161649 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.161616    2546 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 30 00:43:32.161761 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.161689    2546 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 30 00:43:32.161761 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.161698    2546 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 30 00:43:32.162285 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:32.162266    2546 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 30 00:43:32.162362 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:32.162320    2546 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-85.ec2.internal\" not found"
Jul 30 00:43:32.261685 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.261654    2546 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 00:43:32.262870 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.262684    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 00:43:32.262870 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.262717    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 00:43:32.262870 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.262727    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientPID"
Jul 30 00:43:32.262870 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.262751    2546 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-85.ec2.internal"
Jul 30 00:43:32.271196 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.271123    2546 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-85.ec2.internal"
Jul 30 00:43:32.271196 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:32.271150    2546 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-85.ec2.internal\": node \"ip-10-0-141-85.ec2.internal\" not found"
Jul 30 00:43:32.284416 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:32.284395    2546 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-85.ec2.internal\" not found"
Jul 30 00:43:32.302095 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.302062    2546 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 30 00:43:32.303287 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.303274    2546 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 30 00:43:32.303357 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.303314    2546 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 30 00:43:32.303357 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.303338    2546 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 30 00:43:32.303463 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:32.303376    2546 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 30 00:43:32.306595 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:32.306579    2546 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 00:43:33.433439 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:33.433261    2546 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-85.ec2.internal\" not found"
Jul 30 00:43:33.451731 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.451694    2546 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-85.ec2.internal"]
Jul 30 00:43:33.451813 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.451790    2546 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 00:43:33.453452 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.453430    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 00:43:33.453579 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.453464    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 00:43:33.453579 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.453478    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientPID"
Jul 30 00:43:33.455143 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.455129    2546 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 00:43:33.455258 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.455245    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal"
Jul 30 00:43:33.455306 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.455274    2546 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 00:43:33.455941 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.455920    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 00:43:33.456030 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.455950    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 00:43:33.456030 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.455964    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientPID"
Jul 30 00:43:33.456104 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.455965    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 00:43:33.456136 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.456104    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 00:43:33.456136 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.456117    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientPID"
Jul 30 00:43:33.457390 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.457374    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-85.ec2.internal"
Jul 30 00:43:33.457451 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.457403    2546 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 00:43:33.458063 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.458049    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 00:43:33.458132 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.458069    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 00:43:33.458132 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.458083    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientPID"
Jul 30 00:43:33.486267 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:33.486242    2546 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-85.ec2.internal\" not found" node="ip-10-0-141-85.ec2.internal"
Jul 30 00:43:33.490561 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:33.490545    2546 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-85.ec2.internal\" not found" node="ip-10-0-141-85.ec2.internal"
Jul 30 00:43:33.534352 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:33.534325    2546 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-85.ec2.internal\" not found"
Jul 30 00:43:33.544651 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.544633    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ec5ec4be6f2266139bbde9ed8209c955-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal\" (UID: \"ec5ec4be6f2266139bbde9ed8209c955\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal"
Jul 30 00:43:33.544745 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.544658    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec5ec4be6f2266139bbde9ed8209c955-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal\" (UID: \"ec5ec4be6f2266139bbde9ed8209c955\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal"
Jul 30 00:43:33.544745 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.544677    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/80e212081188b2e37af81e25e05fed3c-config\") pod \"kube-apiserver-proxy-ip-10-0-141-85.ec2.internal\" (UID: \"80e212081188b2e37af81e25e05fed3c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-85.ec2.internal"
Jul 30 00:43:33.634964 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:33.634876    2546 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-85.ec2.internal\" not found"
Jul 30 00:43:33.645299 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.645273    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/80e212081188b2e37af81e25e05fed3c-config\") pod \"kube-apiserver-proxy-ip-10-0-141-85.ec2.internal\" (UID: \"80e212081188b2e37af81e25e05fed3c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-85.ec2.internal"
Jul 30 00:43:33.645360 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.645308    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ec5ec4be6f2266139bbde9ed8209c955-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal\" (UID: \"ec5ec4be6f2266139bbde9ed8209c955\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal"
Jul 30 00:43:33.645360 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.645327    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec5ec4be6f2266139bbde9ed8209c955-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal\" (UID: \"ec5ec4be6f2266139bbde9ed8209c955\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal"
Jul 30 00:43:33.645422 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.645362    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ec5ec4be6f2266139bbde9ed8209c955-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal\" (UID: \"ec5ec4be6f2266139bbde9ed8209c955\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal"
Jul 30 00:43:33.645422 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.645366    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/80e212081188b2e37af81e25e05fed3c-config\") pod \"kube-apiserver-proxy-ip-10-0-141-85.ec2.internal\" (UID: \"80e212081188b2e37af81e25e05fed3c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-85.ec2.internal"
Jul 30 00:43:33.645422 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.645404    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec5ec4be6f2266139bbde9ed8209c955-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal\" (UID: \"ec5ec4be6f2266139bbde9ed8209c955\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal"
Jul 30 00:43:33.735483 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:33.735446    2546 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-85.ec2.internal\" not found"
Jul 30 00:43:33.788974 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.788940    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal"
Jul 30 00:43:33.794451 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:33.794431    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-85.ec2.internal"
Jul 30 00:43:33.836538 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:33.836500    2546 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-85.ec2.internal\" not found"
Jul 30 00:43:33.937098 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:33.937012    2546 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-85.ec2.internal\" not found"
Jul 30 00:43:34.037616 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:34.037576    2546 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-85.ec2.internal\" not found"
Jul 30 00:43:34.088057 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:34.088020    2546 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 30 00:43:34.088583 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:34.088227    2546 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 00:43:34.115181 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:34.115150    2546 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 00:43:34.138030 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:34.137997    2546 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-85.ec2.internal\" not found"
Jul 30 00:43:34.155925 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:34.155884    2546 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-29 00:38:33 +0000 UTC" deadline="2027-12-29 23:58:19.021260948 +0000 UTC"
Jul 30 00:43:34.155925 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:34.155918    2546 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12431h14m44.865345885s"
Jul 30 00:43:34.200389 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:34.200332    2546 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 00:43:34.242473 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:34.242449    2546 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal"
Jul 30 00:43:34.253085 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:34.253063    2546 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 00:43:34.255434 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:34.255418    2546 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-85.ec2.internal"
Jul 30 00:43:34.263552 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:34.263515    2546 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 00:43:34.365767 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:34.365738    2546 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 00:43:34.402258 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:34.402213    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80e212081188b2e37af81e25e05fed3c.slice/crio-74ec234cc96c729062f7796a9628c9d5d97a1cbc633f1ca7b4c2f0b9f10f255b WatchSource:0}: Error finding container 74ec234cc96c729062f7796a9628c9d5d97a1cbc633f1ca7b4c2f0b9f10f255b: Status 404 returned error can't find the container with id 74ec234cc96c729062f7796a9628c9d5d97a1cbc633f1ca7b4c2f0b9f10f255b
Jul 30 00:43:34.408073 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:34.408056    2546 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 00:43:34.525553 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:34.525516    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec5ec4be6f2266139bbde9ed8209c955.slice/crio-24ec452cce0c94d168281eb1db2216877e5463bc3b7d83e94d229393dd171a91 WatchSource:0}: Error finding container 24ec452cce0c94d168281eb1db2216877e5463bc3b7d83e94d229393dd171a91: Status 404 returned error can't find the container with id 24ec452cce0c94d168281eb1db2216877e5463bc3b7d83e94d229393dd171a91
Jul 30 00:43:35.122702 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.122654    2546 apiserver.go:52] "Watching apiserver"
Jul 30 00:43:35.131948 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.130975    2546 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 30 00:43:35.131948 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.131547    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-bwfnn","openshift-multus/network-metrics-daemon-dcwcp","openshift-network-diagnostics/network-check-target-zhf7n","kube-system/global-pull-secret-syncer-9lfz5","kube-system/konnectivity-agent-vkxdr","openshift-image-registry/node-ca-7vhrw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal","openshift-network-operator/iptables-alerter-fmmn7","openshift-ovn-kubernetes/ovnkube-node-fnvt9","kube-system/kube-apiserver-proxy-ip-10-0-141-85.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz","openshift-cluster-node-tuning-operator/tuned-286nx","openshift-dns/node-resolver-st67h","openshift-multus/multus-additional-cni-plugins-lnhk8"]
Jul 30 00:43:35.133352 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.133237    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-st67h"
Jul 30 00:43:35.134731 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.134686    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:43:35.134830 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:35.134769    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcwcp" podUID="57b08e9a-8cca-4cde-8f18-e3abe842ac68"
Jul 30 00:43:35.135723 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.135703    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 30 00:43:35.136039 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.136007    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bq2v9\""
Jul 30 00:43:35.136184 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.136162    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 30 00:43:35.136368 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.136354    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:43:35.136434 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:35.136417    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhf7n" podUID="7368d17a-473f-4725-978f-ee53048b033e"
Jul 30 00:43:35.137615 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.137593    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:35.137712 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:35.137659    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lfz5" podUID="b7286a2e-d6a2-4db7-912a-3ae863862afa"
Jul 30 00:43:35.138980 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.138958    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vkxdr"
Jul 30 00:43:35.140237 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.139600    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7vhrw"
Jul 30 00:43:35.141118 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.140840    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.141741 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.141723    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 30 00:43:35.142439 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.142022    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 30 00:43:35.142439 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.142054    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 30 00:43:35.142439 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.142258    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fmmn7"
Jul 30 00:43:35.142681 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.142259    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 30 00:43:35.143642 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.142890    2546 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 30 00:43:35.143642 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.142994    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 30 00:43:35.143642 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.143102    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-57cbs\""
Jul 30 00:43:35.143642 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.143203    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 30 00:43:35.143642 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.143250    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mws9x\""
Jul 30 00:43:35.143642 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.143302    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-52xds\""
Jul 30 00:43:35.143642 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.143435    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 30 00:43:35.144500 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.143989    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 30 00:43:35.144500 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.144027    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 30 00:43:35.144500 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.144354    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.145772 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.144790    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 30 00:43:35.145772 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.145065    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 30 00:43:35.145772 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.145297    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 30 00:43:35.145772 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.145454    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6nwn8\""
Jul 30 00:43:35.145981 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.145907    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.146995 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.146508    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 30 00:43:35.146995 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.146856    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8m4zf\""
Jul 30 00:43:35.147116 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.147027    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 30 00:43:35.147116 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.147103    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 30 00:43:35.149062 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.148551    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz"
Jul 30 00:43:35.149787 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.149768    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 30 00:43:35.150190 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.150172    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 30 00:43:35.150247 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.150230    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 30 00:43:35.150442 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.150422    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zhkb6\""
Jul 30 00:43:35.150484 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.150468    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 30 00:43:35.151122 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.151104    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 30 00:43:35.151496 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.151479    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 30 00:43:35.151880 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.151863    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.152928 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.152907    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 30 00:43:35.154268 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.154246    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 30 00:43:35.154441 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.154427    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7hwz4\""
Jul 30 00:43:35.154585 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.154254    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lgf47\""
Jul 30 00:43:35.154723 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.154699    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 30 00:43:35.154793 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.154741    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 30 00:43:35.154925 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.154903    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-cni-binary-copy\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.154995 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.154942    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-host-var-lib-kubelet\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.154995 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.154969    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf8vf\" (UniqueName: \"kubernetes.io/projected/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-kube-api-access-jf8vf\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.155096 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155003    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.155096 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155027    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66fl9\" (UniqueName: \"kubernetes.io/projected/e0c213ea-4c65-4b0c-813c-0ae9e69047e1-kube-api-access-66fl9\") pod \"node-resolver-st67h\" (UID: \"e0c213ea-4c65-4b0c-813c-0ae9e69047e1\") " pod="openshift-dns/node-resolver-st67h"
Jul 30 00:43:35.155096 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155051    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-multus-cni-dir\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.155240 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155090    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-host-run-multus-certs\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.155240 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155158    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/86d29ffc-9bf4-4a32-a71f-c30674b1924e-ovn-node-metrics-cert\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.155240 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155186    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7eaeb5a9-feed-476d-8a67-083665883cdf-konnectivity-ca\") pod \"konnectivity-agent-vkxdr\" (UID: \"7eaeb5a9-feed-476d-8a67-083665883cdf\") " pod="kube-system/konnectivity-agent-vkxdr"
Jul 30 00:43:35.155240 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155210    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt5qc\" (UniqueName: \"kubernetes.io/projected/b429caa1-4e6d-4f0c-b426-1e8b75379b8b-kube-api-access-dt5qc\") pod \"node-ca-7vhrw\" (UID: \"b429caa1-4e6d-4f0c-b426-1e8b75379b8b\") " pod="openshift-image-registry/node-ca-7vhrw"
Jul 30 00:43:35.155240 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155232    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-cnibin\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.155469 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155256    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-multus-socket-dir-parent\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.155469 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155281    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-host-var-lib-cni-multus\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.155469 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155302    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-hostroot\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.155469 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155324    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-run-openvswitch\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.155469 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155359    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-host-run-ovn-kubernetes\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.155469 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155389    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7aa8886-d7df-47d0-a71a-ad213aab0796-host-slash\") pod \"iptables-alerter-fmmn7\" (UID: \"d7aa8886-d7df-47d0-a71a-ad213aab0796\") " pod="openshift-network-operator/iptables-alerter-fmmn7"
Jul 30 00:43:35.155469 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155413    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-etc-kubernetes\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.155469 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155437    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-host-kubelet\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.155469 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155459    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-systemd-units\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.155880 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155481    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-etc-openvswitch\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.155880 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155502    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e0c213ea-4c65-4b0c-813c-0ae9e69047e1-hosts-file\") pod \"node-resolver-st67h\" (UID: \"e0c213ea-4c65-4b0c-813c-0ae9e69047e1\") " pod="openshift-dns/node-resolver-st67h"
Jul 30 00:43:35.155880 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155545    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7eaeb5a9-feed-476d-8a67-083665883cdf-agent-certs\") pod \"konnectivity-agent-vkxdr\" (UID: \"7eaeb5a9-feed-476d-8a67-083665883cdf\") " pod="kube-system/konnectivity-agent-vkxdr"
Jul 30 00:43:35.155880 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155583    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-multus-daemon-config\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.155880 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155605    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b429caa1-4e6d-4f0c-b426-1e8b75379b8b-host\") pod \"node-ca-7vhrw\" (UID: \"b429caa1-4e6d-4f0c-b426-1e8b75379b8b\") " pod="openshift-image-registry/node-ca-7vhrw"
Jul 30 00:43:35.155880 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155627    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-os-release\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.155880 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155652    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-host-run-netns\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.155880 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155680    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-host-cni-bin\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.155880 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155704    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/86d29ffc-9bf4-4a32-a71f-c30674b1924e-ovnkube-script-lib\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.155880 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155727    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c2nn\" (UniqueName: \"kubernetes.io/projected/57b08e9a-8cca-4cde-8f18-e3abe842ac68-kube-api-access-4c2nn\") pod \"network-metrics-daemon-dcwcp\" (UID: \"57b08e9a-8cca-4cde-8f18-e3abe842ac68\") " pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:43:35.155880 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155749    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-host-var-lib-cni-bin\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.155880 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155772    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-host-cni-netd\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.155880 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155795    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/86d29ffc-9bf4-4a32-a71f-c30674b1924e-ovnkube-config\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.155880 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155859    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b7286a2e-d6a2-4db7-912a-3ae863862afa-kubelet-config\") pod \"global-pull-secret-syncer-9lfz5\" (UID: \"b7286a2e-d6a2-4db7-912a-3ae863862afa\") " pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:35.155880 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155884    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-run-ovn\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.156452 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155906    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-multus-conf-dir\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.156452 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155929    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkk7n\" (UniqueName: \"kubernetes.io/projected/d7aa8886-d7df-47d0-a71a-ad213aab0796-kube-api-access-kkk7n\") pod \"iptables-alerter-fmmn7\" (UID: \"d7aa8886-d7df-47d0-a71a-ad213aab0796\") " pod="openshift-network-operator/iptables-alerter-fmmn7"
Jul 30 00:43:35.156452 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.155960    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-host-run-k8s-cni-cncf-io\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.156452 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.156004    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-host-run-netns\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.156452 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.156031    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-run-systemd\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.156452 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.156052    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b429caa1-4e6d-4f0c-b426-1e8b75379b8b-serviceca\") pod \"node-ca-7vhrw\" (UID: \"b429caa1-4e6d-4f0c-b426-1e8b75379b8b\") " pod="openshift-image-registry/node-ca-7vhrw"
Jul 30 00:43:35.156452 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.156080    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5f68\" (UniqueName: \"kubernetes.io/projected/7368d17a-473f-4725-978f-ee53048b033e-kube-api-access-f5f68\") pod \"network-check-target-zhf7n\" (UID: \"7368d17a-473f-4725-978f-ee53048b033e\") " pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:43:35.156452 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.156103    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-system-cni-dir\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.156452 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.156127    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0c213ea-4c65-4b0c-813c-0ae9e69047e1-tmp-dir\") pod \"node-resolver-st67h\" (UID: \"e0c213ea-4c65-4b0c-813c-0ae9e69047e1\") " pod="openshift-dns/node-resolver-st67h"
Jul 30 00:43:35.156452 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.156148    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-host-slash\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.156452 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.156181    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-log-socket\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.156452 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.156207    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx7tt\" (UniqueName: \"kubernetes.io/projected/86d29ffc-9bf4-4a32-a71f-c30674b1924e-kube-api-access-wx7tt\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.156452 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.156251    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b7286a2e-d6a2-4db7-912a-3ae863862afa-dbus\") pod \"global-pull-secret-syncer-9lfz5\" (UID: \"b7286a2e-d6a2-4db7-912a-3ae863862afa\") " pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:35.156452 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.156283    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/86d29ffc-9bf4-4a32-a71f-c30674b1924e-env-overrides\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.156452 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.156312    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-var-lib-openvswitch\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.156452 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.156339    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-node-log\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.156452 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.156362    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57b08e9a-8cca-4cde-8f18-e3abe842ac68-metrics-certs\") pod \"network-metrics-daemon-dcwcp\" (UID: \"57b08e9a-8cca-4cde-8f18-e3abe842ac68\") " pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:43:35.157230 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.156387    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7286a2e-d6a2-4db7-912a-3ae863862afa-original-pull-secret\") pod \"global-pull-secret-syncer-9lfz5\" (UID: \"b7286a2e-d6a2-4db7-912a-3ae863862afa\") " pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:35.157230 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.156412    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d7aa8886-d7df-47d0-a71a-ad213aab0796-iptables-alerter-script\") pod \"iptables-alerter-fmmn7\" (UID: \"d7aa8886-d7df-47d0-a71a-ad213aab0796\") " pod="openshift-network-operator/iptables-alerter-fmmn7"
Jul 30 00:43:35.159394 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.159347    2546 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 00:43:35.184389 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.184357    2546 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rhnfv"
Jul 30 00:43:35.191016 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.190991    2546 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rhnfv"
Jul 30 00:43:35.243213 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.243185    2546 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 30 00:43:35.256766 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.256732    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-host-var-lib-cni-bin\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.256766 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.256764    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-host-cni-netd\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.257017 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.256781    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/86d29ffc-9bf4-4a32-a71f-c30674b1924e-ovnkube-config\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.257017 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.256802    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b7286a2e-d6a2-4db7-912a-3ae863862afa-kubelet-config\") pod \"global-pull-secret-syncer-9lfz5\" (UID: \"b7286a2e-d6a2-4db7-912a-3ae863862afa\") " pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:35.257017 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.256826    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/49cd90ba-3fc8-4075-83ec-20aa08534f75-device-dir\") pod \"aws-ebs-csi-driver-node-b6nwz\" (UID: \"49cd90ba-3fc8-4075-83ec-20aa08534f75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz"
Jul 30 00:43:35.257017 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.256843    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-run-ovn\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.257017 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.256860    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e419b701-ba27-4810-9851-ae5242cdc2c5-cnibin\") pod \"multus-additional-cni-plugins-lnhk8\" (UID: \"e419b701-ba27-4810-9851-ae5242cdc2c5\") " pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.257017 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.256868    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-host-var-lib-cni-bin\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.257017 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.256876    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-multus-conf-dir\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.257017 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.256913    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-multus-conf-dir\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.257017 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.256930    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkk7n\" (UniqueName: \"kubernetes.io/projected/d7aa8886-d7df-47d0-a71a-ad213aab0796-kube-api-access-kkk7n\") pod \"iptables-alerter-fmmn7\" (UID: \"d7aa8886-d7df-47d0-a71a-ad213aab0796\") " pod="openshift-network-operator/iptables-alerter-fmmn7"
Jul 30 00:43:35.257017 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.256947    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-host-cni-netd\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.257017 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.256963    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e419b701-ba27-4810-9851-ae5242cdc2c5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lnhk8\" (UID: \"e419b701-ba27-4810-9851-ae5242cdc2c5\") " pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.257017 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.256994    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-host-run-k8s-cni-cncf-io\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.257017 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257021    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-host-run-netns\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.257693 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257045    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-run-systemd\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.257693 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257067    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b429caa1-4e6d-4f0c-b426-1e8b75379b8b-serviceca\") pod \"node-ca-7vhrw\" (UID: \"b429caa1-4e6d-4f0c-b426-1e8b75379b8b\") " pod="openshift-image-registry/node-ca-7vhrw"
Jul 30 00:43:35.257693 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257102    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/49cd90ba-3fc8-4075-83ec-20aa08534f75-registration-dir\") pod \"aws-ebs-csi-driver-node-b6nwz\" (UID: \"49cd90ba-3fc8-4075-83ec-20aa08534f75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz"
Jul 30 00:43:35.257693 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257129    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5f68\" (UniqueName: \"kubernetes.io/projected/7368d17a-473f-4725-978f-ee53048b033e-kube-api-access-f5f68\") pod \"network-check-target-zhf7n\" (UID: \"7368d17a-473f-4725-978f-ee53048b033e\") " pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:43:35.257693 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257153    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-system-cni-dir\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.257693 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257192    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/49cd90ba-3fc8-4075-83ec-20aa08534f75-sys-fs\") pod \"aws-ebs-csi-driver-node-b6nwz\" (UID: \"49cd90ba-3fc8-4075-83ec-20aa08534f75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz"
Jul 30 00:43:35.257693 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257219    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0c213ea-4c65-4b0c-813c-0ae9e69047e1-tmp-dir\") pod \"node-resolver-st67h\" (UID: \"e0c213ea-4c65-4b0c-813c-0ae9e69047e1\") " pod="openshift-dns/node-resolver-st67h"
Jul 30 00:43:35.257693 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257241    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/49cd90ba-3fc8-4075-83ec-20aa08534f75-socket-dir\") pod \"aws-ebs-csi-driver-node-b6nwz\" (UID: \"49cd90ba-3fc8-4075-83ec-20aa08534f75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz"
Jul 30 00:43:35.257693 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257257    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-sys\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.257693 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257277    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-host\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.257693 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257307    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-host-slash\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.257693 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257328    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-log-socket\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.257693 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257344    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx7tt\" (UniqueName: \"kubernetes.io/projected/86d29ffc-9bf4-4a32-a71f-c30674b1924e-kube-api-access-wx7tt\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.257693 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257370    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b7286a2e-d6a2-4db7-912a-3ae863862afa-dbus\") pod \"global-pull-secret-syncer-9lfz5\" (UID: \"b7286a2e-d6a2-4db7-912a-3ae863862afa\") " pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:35.257693 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257389    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e419b701-ba27-4810-9851-ae5242cdc2c5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lnhk8\" (UID: \"e419b701-ba27-4810-9851-ae5242cdc2c5\") " pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.257693 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257410    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-etc-sysctl-conf\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.257693 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257422    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/86d29ffc-9bf4-4a32-a71f-c30674b1924e-ovnkube-config\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.258449 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257430    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-etc-modprobe-d\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.258449 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257455    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-run\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.258449 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257494    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b7286a2e-d6a2-4db7-912a-3ae863862afa-kubelet-config\") pod \"global-pull-secret-syncer-9lfz5\" (UID: \"b7286a2e-d6a2-4db7-912a-3ae863862afa\") " pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:35.258449 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257494    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/86d29ffc-9bf4-4a32-a71f-c30674b1924e-env-overrides\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.258449 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257542    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e419b701-ba27-4810-9851-ae5242cdc2c5-system-cni-dir\") pod \"multus-additional-cni-plugins-lnhk8\" (UID: \"e419b701-ba27-4810-9851-ae5242cdc2c5\") " pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.258449 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257569    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-etc-sysconfig\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.258449 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257591    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-etc-kubernetes\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.258449 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257621    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-var-lib-openvswitch\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.258449 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257644    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-node-log\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.258449 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257669    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57b08e9a-8cca-4cde-8f18-e3abe842ac68-metrics-certs\") pod \"network-metrics-daemon-dcwcp\" (UID: \"57b08e9a-8cca-4cde-8f18-e3abe842ac68\") " pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:43:35.258449 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257696    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7286a2e-d6a2-4db7-912a-3ae863862afa-original-pull-secret\") pod \"global-pull-secret-syncer-9lfz5\" (UID: \"b7286a2e-d6a2-4db7-912a-3ae863862afa\") " pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:35.258449 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257724    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d7aa8886-d7df-47d0-a71a-ad213aab0796-iptables-alerter-script\") pod \"iptables-alerter-fmmn7\" (UID: \"d7aa8886-d7df-47d0-a71a-ad213aab0796\") " pod="openshift-network-operator/iptables-alerter-fmmn7"
Jul 30 00:43:35.258449 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257752    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e419b701-ba27-4810-9851-ae5242cdc2c5-cni-binary-copy\") pod \"multus-additional-cni-plugins-lnhk8\" (UID: \"e419b701-ba27-4810-9851-ae5242cdc2c5\") " pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.258449 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257776    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/49cd90ba-3fc8-4075-83ec-20aa08534f75-etc-selinux\") pod \"aws-ebs-csi-driver-node-b6nwz\" (UID: \"49cd90ba-3fc8-4075-83ec-20aa08534f75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz"
Jul 30 00:43:35.258449 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257805    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-cni-binary-copy\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.258449 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257813    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0c213ea-4c65-4b0c-813c-0ae9e69047e1-tmp-dir\") pod \"node-resolver-st67h\" (UID: \"e0c213ea-4c65-4b0c-813c-0ae9e69047e1\") " pod="openshift-dns/node-resolver-st67h"
Jul 30 00:43:35.258449 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257831    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-host-var-lib-kubelet\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.259105 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257859    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf8vf\" (UniqueName: \"kubernetes.io/projected/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-kube-api-access-jf8vf\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.259105 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257884    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.259105 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257890    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/86d29ffc-9bf4-4a32-a71f-c30674b1924e-env-overrides\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.259105 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257909    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66fl9\" (UniqueName: \"kubernetes.io/projected/e0c213ea-4c65-4b0c-813c-0ae9e69047e1-kube-api-access-66fl9\") pod \"node-resolver-st67h\" (UID: \"e0c213ea-4c65-4b0c-813c-0ae9e69047e1\") " pod="openshift-dns/node-resolver-st67h"
Jul 30 00:43:35.259105 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257933    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-etc-systemd\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.259105 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257957    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-lib-modules\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.259105 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257973    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-host-slash\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.259105 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.257979    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a767f76b-efec-44ec-a0e8-a74c4c589d65-etc-tuned\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.259105 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258007    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-multus-cni-dir\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.259105 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258006    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-log-socket\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.259105 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258035    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-host-run-multus-certs\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.259105 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258080    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-host-run-multus-certs\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.259105 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258221    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/86d29ffc-9bf4-4a32-a71f-c30674b1924e-ovn-node-metrics-cert\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.259105 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258252    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a767f76b-efec-44ec-a0e8-a74c4c589d65-tmp\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.259105 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258278    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7eaeb5a9-feed-476d-8a67-083665883cdf-konnectivity-ca\") pod \"konnectivity-agent-vkxdr\" (UID: \"7eaeb5a9-feed-476d-8a67-083665883cdf\") " pod="kube-system/konnectivity-agent-vkxdr"
Jul 30 00:43:35.259105 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258304    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt5qc\" (UniqueName: \"kubernetes.io/projected/b429caa1-4e6d-4f0c-b426-1e8b75379b8b-kube-api-access-dt5qc\") pod \"node-ca-7vhrw\" (UID: \"b429caa1-4e6d-4f0c-b426-1e8b75379b8b\") " pod="openshift-image-registry/node-ca-7vhrw"
Jul 30 00:43:35.259105 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258329    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-cnibin\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.259732 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258354    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-multus-socket-dir-parent\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.259732 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258381    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-host-var-lib-cni-multus\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.259732 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258404    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-var-lib-openvswitch\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.259732 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258405    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-hostroot\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.259732 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258442    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-hostroot\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.259732 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258442    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-run-openvswitch\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.259732 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258471    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b429caa1-4e6d-4f0c-b426-1e8b75379b8b-serviceca\") pod \"node-ca-7vhrw\" (UID: \"b429caa1-4e6d-4f0c-b426-1e8b75379b8b\") " pod="openshift-image-registry/node-ca-7vhrw"
Jul 30 00:43:35.259732 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258477    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-host-run-ovn-kubernetes\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.259732 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258475    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-run-openvswitch\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.259732 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258505    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7aa8886-d7df-47d0-a71a-ad213aab0796-host-slash\") pod \"iptables-alerter-fmmn7\" (UID: \"d7aa8886-d7df-47d0-a71a-ad213aab0796\") " pod="openshift-network-operator/iptables-alerter-fmmn7"
Jul 30 00:43:35.259732 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258512    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-node-log\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.259732 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258558    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-etc-kubernetes\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.259732 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258563    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-host-run-k8s-cni-cncf-io\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.259732 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258609    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-etc-kubernetes\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.259732 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:35.258788    2546 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 00:43:35.259732 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258820    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-cni-binary-copy\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.259732 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:35.258861    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7286a2e-d6a2-4db7-912a-3ae863862afa-original-pull-secret podName:b7286a2e-d6a2-4db7-912a-3ae863862afa nodeName:}" failed. No retries permitted until 2026-07-30 00:43:35.758832456 +0000 UTC m=+2.981566277 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b7286a2e-d6a2-4db7-912a-3ae863862afa-original-pull-secret") pod "global-pull-secret-syncer-9lfz5" (UID: "b7286a2e-d6a2-4db7-912a-3ae863862afa") : object "kube-system"/"original-pull-secret" not registered
Jul 30 00:43:35.259732 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258870    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-host-var-lib-kubelet\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.260481 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.258899    2546 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 30 00:43:35.260481 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:35.258990    2546 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:43:35.260481 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259066    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-multus-cni-dir\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.260481 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259086    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-run-systemd\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.260481 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259141    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-run-ovn\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.260481 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259204    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-system-cni-dir\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.260481 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259287    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.260481 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259342    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-cnibin\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.260481 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259368    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b7286a2e-d6a2-4db7-912a-3ae863862afa-dbus\") pod \"global-pull-secret-syncer-9lfz5\" (UID: \"b7286a2e-d6a2-4db7-912a-3ae863862afa\") " pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:35.260481 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259408    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-host-var-lib-cni-multus\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.260481 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259410    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-host-run-ovn-kubernetes\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.260481 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259440    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7aa8886-d7df-47d0-a71a-ad213aab0796-host-slash\") pod \"iptables-alerter-fmmn7\" (UID: \"d7aa8886-d7df-47d0-a71a-ad213aab0796\") " pod="openshift-network-operator/iptables-alerter-fmmn7"
Jul 30 00:43:35.260481 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:35.259018    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57b08e9a-8cca-4cde-8f18-e3abe842ac68-metrics-certs podName:57b08e9a-8cca-4cde-8f18-e3abe842ac68 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:35.759009033 +0000 UTC m=+2.981742830 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57b08e9a-8cca-4cde-8f18-e3abe842ac68-metrics-certs") pod "network-metrics-daemon-dcwcp" (UID: "57b08e9a-8cca-4cde-8f18-e3abe842ac68") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:43:35.260481 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259483    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-host-kubelet\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.260481 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259488    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-multus-socket-dir-parent\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.260481 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259543    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-host-run-netns\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.260481 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259566    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-systemd-units\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.260481 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259595    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-host-kubelet\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.261392 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259635    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-etc-openvswitch\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.261392 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259662    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e0c213ea-4c65-4b0c-813c-0ae9e69047e1-hosts-file\") pod \"node-resolver-st67h\" (UID: \"e0c213ea-4c65-4b0c-813c-0ae9e69047e1\") " pod="openshift-dns/node-resolver-st67h"
Jul 30 00:43:35.261392 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259689    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7eaeb5a9-feed-476d-8a67-083665883cdf-agent-certs\") pod \"konnectivity-agent-vkxdr\" (UID: \"7eaeb5a9-feed-476d-8a67-083665883cdf\") " pod="kube-system/konnectivity-agent-vkxdr"
Jul 30 00:43:35.261392 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259691    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-systemd-units\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.261392 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259739    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/49cd90ba-3fc8-4075-83ec-20aa08534f75-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b6nwz\" (UID: \"49cd90ba-3fc8-4075-83ec-20aa08534f75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz"
Jul 30 00:43:35.261392 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259748    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-etc-openvswitch\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.261392 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259798    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-multus-daemon-config\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.261392 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259839    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b429caa1-4e6d-4f0c-b426-1e8b75379b8b-host\") pod \"node-ca-7vhrw\" (UID: \"b429caa1-4e6d-4f0c-b426-1e8b75379b8b\") " pod="openshift-image-registry/node-ca-7vhrw"
Jul 30 00:43:35.261392 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259868    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e419b701-ba27-4810-9851-ae5242cdc2c5-os-release\") pod \"multus-additional-cni-plugins-lnhk8\" (UID: \"e419b701-ba27-4810-9851-ae5242cdc2c5\") " pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.261392 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259870    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e0c213ea-4c65-4b0c-813c-0ae9e69047e1-hosts-file\") pod \"node-resolver-st67h\" (UID: \"e0c213ea-4c65-4b0c-813c-0ae9e69047e1\") " pod="openshift-dns/node-resolver-st67h"
Jul 30 00:43:35.261392 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259913    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e419b701-ba27-4810-9851-ae5242cdc2c5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lnhk8\" (UID: \"e419b701-ba27-4810-9851-ae5242cdc2c5\") " pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.261392 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259916    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b429caa1-4e6d-4f0c-b426-1e8b75379b8b-host\") pod \"node-ca-7vhrw\" (UID: \"b429caa1-4e6d-4f0c-b426-1e8b75379b8b\") " pod="openshift-image-registry/node-ca-7vhrw"
Jul 30 00:43:35.261392 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259946    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv9w7\" (UniqueName: \"kubernetes.io/projected/e419b701-ba27-4810-9851-ae5242cdc2c5-kube-api-access-hv9w7\") pod \"multus-additional-cni-plugins-lnhk8\" (UID: \"e419b701-ba27-4810-9851-ae5242cdc2c5\") " pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.261392 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.259990    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7eaeb5a9-feed-476d-8a67-083665883cdf-konnectivity-ca\") pod \"konnectivity-agent-vkxdr\" (UID: \"7eaeb5a9-feed-476d-8a67-083665883cdf\") " pod="kube-system/konnectivity-agent-vkxdr"
Jul 30 00:43:35.261392 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.260037    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-etc-sysctl-d\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.261392 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.260074    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-var-lib-kubelet\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.261392 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.260111    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-os-release\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.262180 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.260138    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-host-run-netns\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.262180 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.260168    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-host-cni-bin\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.262180 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.260176    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d7aa8886-d7df-47d0-a71a-ad213aab0796-iptables-alerter-script\") pod \"iptables-alerter-fmmn7\" (UID: \"d7aa8886-d7df-47d0-a71a-ad213aab0796\") " pod="openshift-network-operator/iptables-alerter-fmmn7"
Jul 30 00:43:35.262180 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.260189    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-os-release\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.262180 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.260184    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/86d29ffc-9bf4-4a32-a71f-c30674b1924e-ovnkube-script-lib\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.262180 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.260256    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c2nn\" (UniqueName: \"kubernetes.io/projected/57b08e9a-8cca-4cde-8f18-e3abe842ac68-kube-api-access-4c2nn\") pod \"network-metrics-daemon-dcwcp\" (UID: \"57b08e9a-8cca-4cde-8f18-e3abe842ac68\") " pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:43:35.262180 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.260260    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-host-run-netns\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.262180 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.260302    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/86d29ffc-9bf4-4a32-a71f-c30674b1924e-host-cni-bin\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.262180 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.260347    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-multus-daemon-config\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.262180 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.260403    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpcdw\" (UniqueName: \"kubernetes.io/projected/49cd90ba-3fc8-4075-83ec-20aa08534f75-kube-api-access-bpcdw\") pod \"aws-ebs-csi-driver-node-b6nwz\" (UID: \"49cd90ba-3fc8-4075-83ec-20aa08534f75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz"
Jul 30 00:43:35.262180 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.260433    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9lp7\" (UniqueName: \"kubernetes.io/projected/a767f76b-efec-44ec-a0e8-a74c4c589d65-kube-api-access-b9lp7\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.262180 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.260610    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/86d29ffc-9bf4-4a32-a71f-c30674b1924e-ovnkube-script-lib\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.264210 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.264070    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/86d29ffc-9bf4-4a32-a71f-c30674b1924e-ovn-node-metrics-cert\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.265924 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.265898    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7eaeb5a9-feed-476d-8a67-083665883cdf-agent-certs\") pod \"konnectivity-agent-vkxdr\" (UID: \"7eaeb5a9-feed-476d-8a67-083665883cdf\") " pod="kube-system/konnectivity-agent-vkxdr"
Jul 30 00:43:35.265924 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:35.265915    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 00:43:35.266097 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:35.265939    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 00:43:35.266097 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:35.265952    2546 projected.go:196] Error preparing data for projected volume kube-api-access-f5f68 for pod openshift-network-diagnostics/network-check-target-zhf7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:43:35.266750 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:35.266497    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7368d17a-473f-4725-978f-ee53048b033e-kube-api-access-f5f68 podName:7368d17a-473f-4725-978f-ee53048b033e nodeName:}" failed. No retries permitted until 2026-07-30 00:43:35.766471628 +0000 UTC m=+2.989205441 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-f5f68" (UniqueName: "kubernetes.io/projected/7368d17a-473f-4725-978f-ee53048b033e-kube-api-access-f5f68") pod "network-check-target-zhf7n" (UID: "7368d17a-473f-4725-978f-ee53048b033e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:43:35.270215 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.269725    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt5qc\" (UniqueName: \"kubernetes.io/projected/b429caa1-4e6d-4f0c-b426-1e8b75379b8b-kube-api-access-dt5qc\") pod \"node-ca-7vhrw\" (UID: \"b429caa1-4e6d-4f0c-b426-1e8b75379b8b\") " pod="openshift-image-registry/node-ca-7vhrw"
Jul 30 00:43:35.270215 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.269801    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf8vf\" (UniqueName: \"kubernetes.io/projected/9fd5e9cd-5cde-452d-b349-31a4c1a12ec9-kube-api-access-jf8vf\") pod \"multus-bwfnn\" (UID: \"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9\") " pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.270371 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.270356    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66fl9\" (UniqueName: \"kubernetes.io/projected/e0c213ea-4c65-4b0c-813c-0ae9e69047e1-kube-api-access-66fl9\") pod \"node-resolver-st67h\" (UID: \"e0c213ea-4c65-4b0c-813c-0ae9e69047e1\") " pod="openshift-dns/node-resolver-st67h"
Jul 30 00:43:35.270985 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.270964    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx7tt\" (UniqueName: \"kubernetes.io/projected/86d29ffc-9bf4-4a32-a71f-c30674b1924e-kube-api-access-wx7tt\") pod \"ovnkube-node-fnvt9\" (UID: \"86d29ffc-9bf4-4a32-a71f-c30674b1924e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.274578 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.274507    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c2nn\" (UniqueName: \"kubernetes.io/projected/57b08e9a-8cca-4cde-8f18-e3abe842ac68-kube-api-access-4c2nn\") pod \"network-metrics-daemon-dcwcp\" (UID: \"57b08e9a-8cca-4cde-8f18-e3abe842ac68\") " pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:43:35.275296 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.275276    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkk7n\" (UniqueName: \"kubernetes.io/projected/d7aa8886-d7df-47d0-a71a-ad213aab0796-kube-api-access-kkk7n\") pod \"iptables-alerter-fmmn7\" (UID: \"d7aa8886-d7df-47d0-a71a-ad213aab0796\") " pod="openshift-network-operator/iptables-alerter-fmmn7"
Jul 30 00:43:35.320301 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.320271    2546 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 00:43:35.356718 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.356664    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal" event={"ID":"ec5ec4be6f2266139bbde9ed8209c955","Type":"ContainerStarted","Data":"24ec452cce0c94d168281eb1db2216877e5463bc3b7d83e94d229393dd171a91"}
Jul 30 00:43:35.358261 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.358231    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-85.ec2.internal" event={"ID":"80e212081188b2e37af81e25e05fed3c","Type":"ContainerStarted","Data":"74ec234cc96c729062f7796a9628c9d5d97a1cbc633f1ca7b4c2f0b9f10f255b"}
Jul 30 00:43:35.361686 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.361648    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e419b701-ba27-4810-9851-ae5242cdc2c5-os-release\") pod \"multus-additional-cni-plugins-lnhk8\" (UID: \"e419b701-ba27-4810-9851-ae5242cdc2c5\") " pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.361808 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.361687    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e419b701-ba27-4810-9851-ae5242cdc2c5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lnhk8\" (UID: \"e419b701-ba27-4810-9851-ae5242cdc2c5\") " pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.361808 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.361715    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv9w7\" (UniqueName: \"kubernetes.io/projected/e419b701-ba27-4810-9851-ae5242cdc2c5-kube-api-access-hv9w7\") pod \"multus-additional-cni-plugins-lnhk8\" (UID: \"e419b701-ba27-4810-9851-ae5242cdc2c5\") " pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.361808 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.361744    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-etc-sysctl-d\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.361808 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.361784    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-var-lib-kubelet\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.361994 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.361816    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpcdw\" (UniqueName: \"kubernetes.io/projected/49cd90ba-3fc8-4075-83ec-20aa08534f75-kube-api-access-bpcdw\") pod \"aws-ebs-csi-driver-node-b6nwz\" (UID: \"49cd90ba-3fc8-4075-83ec-20aa08534f75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz"
Jul 30 00:43:35.361994 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.361842    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9lp7\" (UniqueName: \"kubernetes.io/projected/a767f76b-efec-44ec-a0e8-a74c4c589d65-kube-api-access-b9lp7\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.361994 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.361869    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/49cd90ba-3fc8-4075-83ec-20aa08534f75-device-dir\") pod \"aws-ebs-csi-driver-node-b6nwz\" (UID: \"49cd90ba-3fc8-4075-83ec-20aa08534f75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz"
Jul 30 00:43:35.361994 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.361894    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e419b701-ba27-4810-9851-ae5242cdc2c5-cnibin\") pod \"multus-additional-cni-plugins-lnhk8\" (UID: \"e419b701-ba27-4810-9851-ae5242cdc2c5\") " pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.361994 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.361922    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e419b701-ba27-4810-9851-ae5242cdc2c5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lnhk8\" (UID: \"e419b701-ba27-4810-9851-ae5242cdc2c5\") " pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.361994 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.361946    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/49cd90ba-3fc8-4075-83ec-20aa08534f75-registration-dir\") pod \"aws-ebs-csi-driver-node-b6nwz\" (UID: \"49cd90ba-3fc8-4075-83ec-20aa08534f75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz"
Jul 30 00:43:35.361994 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.361977    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/49cd90ba-3fc8-4075-83ec-20aa08534f75-sys-fs\") pod \"aws-ebs-csi-driver-node-b6nwz\" (UID: \"49cd90ba-3fc8-4075-83ec-20aa08534f75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz"
Jul 30 00:43:35.361994 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.361994    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/49cd90ba-3fc8-4075-83ec-20aa08534f75-socket-dir\") pod \"aws-ebs-csi-driver-node-b6nwz\" (UID: \"49cd90ba-3fc8-4075-83ec-20aa08534f75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz"
Jul 30 00:43:35.362391 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362011    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-sys\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.362391 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362032    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-host\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.362391 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362056    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e419b701-ba27-4810-9851-ae5242cdc2c5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lnhk8\" (UID: \"e419b701-ba27-4810-9851-ae5242cdc2c5\") " pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.362391 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362073    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-etc-sysctl-conf\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.362391 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362099    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-etc-modprobe-d\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.362391 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362113    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-run\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.362391 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362128    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e419b701-ba27-4810-9851-ae5242cdc2c5-system-cni-dir\") pod \"multus-additional-cni-plugins-lnhk8\" (UID: \"e419b701-ba27-4810-9851-ae5242cdc2c5\") " pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.362391 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362148    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-etc-sysconfig\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.362391 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362163    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-etc-kubernetes\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.362391 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362171    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e419b701-ba27-4810-9851-ae5242cdc2c5-os-release\") pod \"multus-additional-cni-plugins-lnhk8\" (UID: \"e419b701-ba27-4810-9851-ae5242cdc2c5\") " pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.362391 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362195    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e419b701-ba27-4810-9851-ae5242cdc2c5-cni-binary-copy\") pod \"multus-additional-cni-plugins-lnhk8\" (UID: \"e419b701-ba27-4810-9851-ae5242cdc2c5\") " pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.362391 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362211    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/49cd90ba-3fc8-4075-83ec-20aa08534f75-etc-selinux\") pod \"aws-ebs-csi-driver-node-b6nwz\" (UID: \"49cd90ba-3fc8-4075-83ec-20aa08534f75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz"
Jul 30 00:43:35.362391 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362228    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-etc-systemd\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.362391 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362253    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-lib-modules\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.362391 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362271    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a767f76b-efec-44ec-a0e8-a74c4c589d65-etc-tuned\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.362391 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362291    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a767f76b-efec-44ec-a0e8-a74c4c589d65-tmp\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.362391 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362342    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/49cd90ba-3fc8-4075-83ec-20aa08534f75-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b6nwz\" (UID: \"49cd90ba-3fc8-4075-83ec-20aa08534f75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz"
Jul 30 00:43:35.363219 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362362    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e419b701-ba27-4810-9851-ae5242cdc2c5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lnhk8\" (UID: \"e419b701-ba27-4810-9851-ae5242cdc2c5\") " pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.363219 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362415    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/49cd90ba-3fc8-4075-83ec-20aa08534f75-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b6nwz\" (UID: \"49cd90ba-3fc8-4075-83ec-20aa08534f75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz"
Jul 30 00:43:35.363219 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362509    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-etc-sysctl-conf\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.363219 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362634    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-etc-modprobe-d\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.363219 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362670    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-run\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.363219 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362695    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e419b701-ba27-4810-9851-ae5242cdc2c5-system-cni-dir\") pod \"multus-additional-cni-plugins-lnhk8\" (UID: \"e419b701-ba27-4810-9851-ae5242cdc2c5\") " pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.363219 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362723    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-etc-sysconfig\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.363219 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.362761    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-etc-kubernetes\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.363219 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.363063    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-etc-sysctl-d\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.363219 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.363129    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/49cd90ba-3fc8-4075-83ec-20aa08534f75-registration-dir\") pod \"aws-ebs-csi-driver-node-b6nwz\" (UID: \"49cd90ba-3fc8-4075-83ec-20aa08534f75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz"
Jul 30 00:43:35.363219 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.363168    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e419b701-ba27-4810-9851-ae5242cdc2c5-cni-binary-copy\") pod \"multus-additional-cni-plugins-lnhk8\" (UID: \"e419b701-ba27-4810-9851-ae5242cdc2c5\") " pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.363219 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.363185    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/49cd90ba-3fc8-4075-83ec-20aa08534f75-sys-fs\") pod \"aws-ebs-csi-driver-node-b6nwz\" (UID: \"49cd90ba-3fc8-4075-83ec-20aa08534f75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz"
Jul 30 00:43:35.363788 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.363245    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-etc-systemd\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.363788 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.363291    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/49cd90ba-3fc8-4075-83ec-20aa08534f75-socket-dir\") pod \"aws-ebs-csi-driver-node-b6nwz\" (UID: \"49cd90ba-3fc8-4075-83ec-20aa08534f75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz"
Jul 30 00:43:35.363788 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.363340    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-lib-modules\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.363788 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.363373    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/49cd90ba-3fc8-4075-83ec-20aa08534f75-etc-selinux\") pod \"aws-ebs-csi-driver-node-b6nwz\" (UID: \"49cd90ba-3fc8-4075-83ec-20aa08534f75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz"
Jul 30 00:43:35.363788 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.363426    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-sys\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.363788 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.363475    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-host\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.363788 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.363540    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e419b701-ba27-4810-9851-ae5242cdc2c5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lnhk8\" (UID: \"e419b701-ba27-4810-9851-ae5242cdc2c5\") " pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.363788 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.363568    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e419b701-ba27-4810-9851-ae5242cdc2c5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lnhk8\" (UID: \"e419b701-ba27-4810-9851-ae5242cdc2c5\") " pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.363788 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.363602    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e419b701-ba27-4810-9851-ae5242cdc2c5-cnibin\") pod \"multus-additional-cni-plugins-lnhk8\" (UID: \"e419b701-ba27-4810-9851-ae5242cdc2c5\") " pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.363788 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.363644    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a767f76b-efec-44ec-a0e8-a74c4c589d65-var-lib-kubelet\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.363788 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.363773    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/49cd90ba-3fc8-4075-83ec-20aa08534f75-device-dir\") pod \"aws-ebs-csi-driver-node-b6nwz\" (UID: \"49cd90ba-3fc8-4075-83ec-20aa08534f75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz"
Jul 30 00:43:35.366121 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.366037    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a767f76b-efec-44ec-a0e8-a74c4c589d65-tmp\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.366239 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.366169    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a767f76b-efec-44ec-a0e8-a74c4c589d65-etc-tuned\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.371097 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.371074    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv9w7\" (UniqueName: \"kubernetes.io/projected/e419b701-ba27-4810-9851-ae5242cdc2c5-kube-api-access-hv9w7\") pod \"multus-additional-cni-plugins-lnhk8\" (UID: \"e419b701-ba27-4810-9851-ae5242cdc2c5\") " pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.371868 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.371845    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpcdw\" (UniqueName: \"kubernetes.io/projected/49cd90ba-3fc8-4075-83ec-20aa08534f75-kube-api-access-bpcdw\") pod \"aws-ebs-csi-driver-node-b6nwz\" (UID: \"49cd90ba-3fc8-4075-83ec-20aa08534f75\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz"
Jul 30 00:43:35.373402 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.373344    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9lp7\" (UniqueName: \"kubernetes.io/projected/a767f76b-efec-44ec-a0e8-a74c4c589d65-kube-api-access-b9lp7\") pod \"tuned-286nx\" (UID: \"a767f76b-efec-44ec-a0e8-a74c4c589d65\") " pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.457336 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.457290    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-st67h"
Jul 30 00:43:35.465253 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.465228    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vkxdr"
Jul 30 00:43:35.474999 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.474971    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7vhrw"
Jul 30 00:43:35.481740 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.481715    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bwfnn"
Jul 30 00:43:35.489579 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.489549    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fmmn7"
Jul 30 00:43:35.499364 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.499339    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:43:35.508219 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.508189    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lnhk8"
Jul 30 00:43:35.516001 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.515975    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz"
Jul 30 00:43:35.522979 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.522957    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-286nx"
Jul 30 00:43:35.765767 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.765691    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57b08e9a-8cca-4cde-8f18-e3abe842ac68-metrics-certs\") pod \"network-metrics-daemon-dcwcp\" (UID: \"57b08e9a-8cca-4cde-8f18-e3abe842ac68\") " pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:43:35.765767 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.765721    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7286a2e-d6a2-4db7-912a-3ae863862afa-original-pull-secret\") pod \"global-pull-secret-syncer-9lfz5\" (UID: \"b7286a2e-d6a2-4db7-912a-3ae863862afa\") " pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:35.765941 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:35.765830    2546 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 00:43:35.765941 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:35.765832    2546 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:43:35.765941 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:35.765880    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7286a2e-d6a2-4db7-912a-3ae863862afa-original-pull-secret podName:b7286a2e-d6a2-4db7-912a-3ae863862afa nodeName:}" failed. No retries permitted until 2026-07-30 00:43:36.765865808 +0000 UTC m=+3.988599617 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b7286a2e-d6a2-4db7-912a-3ae863862afa-original-pull-secret") pod "global-pull-secret-syncer-9lfz5" (UID: "b7286a2e-d6a2-4db7-912a-3ae863862afa") : object "kube-system"/"original-pull-secret" not registered
Jul 30 00:43:35.765941 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:35.765892    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57b08e9a-8cca-4cde-8f18-e3abe842ac68-metrics-certs podName:57b08e9a-8cca-4cde-8f18-e3abe842ac68 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:36.765886407 +0000 UTC m=+3.988620202 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57b08e9a-8cca-4cde-8f18-e3abe842ac68-metrics-certs") pod "network-metrics-daemon-dcwcp" (UID: "57b08e9a-8cca-4cde-8f18-e3abe842ac68") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:43:35.866946 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:35.866919    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5f68\" (UniqueName: \"kubernetes.io/projected/7368d17a-473f-4725-978f-ee53048b033e-kube-api-access-f5f68\") pod \"network-check-target-zhf7n\" (UID: \"7368d17a-473f-4725-978f-ee53048b033e\") " pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:43:35.867121 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:35.867039    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 00:43:35.867121 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:35.867057    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 00:43:35.867121 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:35.867067    2546 projected.go:196] Error preparing data for projected volume kube-api-access-f5f68 for pod openshift-network-diagnostics/network-check-target-zhf7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:43:35.867121 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:35.867119    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7368d17a-473f-4725-978f-ee53048b033e-kube-api-access-f5f68 podName:7368d17a-473f-4725-978f-ee53048b033e nodeName:}" failed. No retries permitted until 2026-07-30 00:43:36.867101362 +0000 UTC m=+4.089835158 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-f5f68" (UniqueName: "kubernetes.io/projected/7368d17a-473f-4725-978f-ee53048b033e-kube-api-access-f5f68") pod "network-check-target-zhf7n" (UID: "7368d17a-473f-4725-978f-ee53048b033e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:43:36.193993 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:36.193891    2546 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 00:38:35 +0000 UTC" deadline="2028-02-18 23:03:57.706267275 +0000 UTC"
Jul 30 00:43:36.193993 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:36.193932    2546 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13654h20m21.512339531s"
Jul 30 00:43:36.353005 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:36.352971    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7aa8886_d7df_47d0_a71a_ad213aab0796.slice/crio-8cdb505d740b03ce3f51e9545d5b32e7acfaebe9a219332c329b1b436b2c486a WatchSource:0}: Error finding container 8cdb505d740b03ce3f51e9545d5b32e7acfaebe9a219332c329b1b436b2c486a: Status 404 returned error can't find the container with id 8cdb505d740b03ce3f51e9545d5b32e7acfaebe9a219332c329b1b436b2c486a
Jul 30 00:43:36.353998 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:36.353868    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86d29ffc_9bf4_4a32_a71f_c30674b1924e.slice/crio-57fb8143ffd3fade7194a127d5ae3b97c6c74608c0232eeb6e620f8094091767 WatchSource:0}: Error finding container 57fb8143ffd3fade7194a127d5ae3b97c6c74608c0232eeb6e620f8094091767: Status 404 returned error can't find the container with id 57fb8143ffd3fade7194a127d5ae3b97c6c74608c0232eeb6e620f8094091767
Jul 30 00:43:36.357096 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:36.356993    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda767f76b_efec_44ec_a0e8_a74c4c589d65.slice/crio-f7279f6482f13b1a6b927127e1f34606494937afc1ba0f2c6d8ebaccc4ed1ab8 WatchSource:0}: Error finding container f7279f6482f13b1a6b927127e1f34606494937afc1ba0f2c6d8ebaccc4ed1ab8: Status 404 returned error can't find the container with id f7279f6482f13b1a6b927127e1f34606494937afc1ba0f2c6d8ebaccc4ed1ab8
Jul 30 00:43:36.364440 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:36.363547    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9" event={"ID":"86d29ffc-9bf4-4a32-a71f-c30674b1924e","Type":"ContainerStarted","Data":"57fb8143ffd3fade7194a127d5ae3b97c6c74608c0232eeb6e620f8094091767"}
Jul 30 00:43:36.370690 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:43:36.370000    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0c213ea_4c65_4b0c_813c_0ae9e69047e1.slice/crio-d909449995472f6c3144516cb4ef7e943adbcc1a2fa4627fe03a06d199366fe1 WatchSource:0}: Error finding container d909449995472f6c3144516cb4ef7e943adbcc1a2fa4627fe03a06d199366fe1: Status 404 returned error can't find the container with id d909449995472f6c3144516cb4ef7e943adbcc1a2fa4627fe03a06d199366fe1
Jul 30 00:43:36.371631 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:36.371603    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fmmn7" event={"ID":"d7aa8886-d7df-47d0-a71a-ad213aab0796","Type":"ContainerStarted","Data":"8cdb505d740b03ce3f51e9545d5b32e7acfaebe9a219332c329b1b436b2c486a"}
Jul 30 00:43:36.373026 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:36.372997    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-286nx" event={"ID":"a767f76b-efec-44ec-a0e8-a74c4c589d65","Type":"ContainerStarted","Data":"f7279f6482f13b1a6b927127e1f34606494937afc1ba0f2c6d8ebaccc4ed1ab8"}
Jul 30 00:43:36.773670 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:36.773336    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57b08e9a-8cca-4cde-8f18-e3abe842ac68-metrics-certs\") pod \"network-metrics-daemon-dcwcp\" (UID: \"57b08e9a-8cca-4cde-8f18-e3abe842ac68\") " pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:43:36.773670 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:36.773627    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7286a2e-d6a2-4db7-912a-3ae863862afa-original-pull-secret\") pod \"global-pull-secret-syncer-9lfz5\" (UID: \"b7286a2e-d6a2-4db7-912a-3ae863862afa\") " pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:36.773882 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:36.773477    2546 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:43:36.773882 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:36.773848    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57b08e9a-8cca-4cde-8f18-e3abe842ac68-metrics-certs podName:57b08e9a-8cca-4cde-8f18-e3abe842ac68 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:38.773828403 +0000 UTC m=+5.996562201 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57b08e9a-8cca-4cde-8f18-e3abe842ac68-metrics-certs") pod "network-metrics-daemon-dcwcp" (UID: "57b08e9a-8cca-4cde-8f18-e3abe842ac68") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:43:36.774185 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:36.773775    2546 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 00:43:36.774273 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:36.774223    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7286a2e-d6a2-4db7-912a-3ae863862afa-original-pull-secret podName:b7286a2e-d6a2-4db7-912a-3ae863862afa nodeName:}" failed. No retries permitted until 2026-07-30 00:43:38.774207536 +0000 UTC m=+5.996941332 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b7286a2e-d6a2-4db7-912a-3ae863862afa-original-pull-secret") pod "global-pull-secret-syncer-9lfz5" (UID: "b7286a2e-d6a2-4db7-912a-3ae863862afa") : object "kube-system"/"original-pull-secret" not registered
Jul 30 00:43:36.874472 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:36.874442    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5f68\" (UniqueName: \"kubernetes.io/projected/7368d17a-473f-4725-978f-ee53048b033e-kube-api-access-f5f68\") pod \"network-check-target-zhf7n\" (UID: \"7368d17a-473f-4725-978f-ee53048b033e\") " pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:43:36.874683 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:36.874664    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 00:43:36.874754 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:36.874692    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 00:43:36.874754 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:36.874706    2546 projected.go:196] Error preparing data for projected volume kube-api-access-f5f68 for pod openshift-network-diagnostics/network-check-target-zhf7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:43:36.874876 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:36.874814    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7368d17a-473f-4725-978f-ee53048b033e-kube-api-access-f5f68 podName:7368d17a-473f-4725-978f-ee53048b033e nodeName:}" failed. No retries permitted until 2026-07-30 00:43:38.874770916 +0000 UTC m=+6.097504731 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-f5f68" (UniqueName: "kubernetes.io/projected/7368d17a-473f-4725-978f-ee53048b033e-kube-api-access-f5f68") pod "network-check-target-zhf7n" (UID: "7368d17a-473f-4725-978f-ee53048b033e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:43:37.194883 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:37.194779    2546 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 00:38:35 +0000 UTC" deadline="2028-04-20 08:43:23.981198266 +0000 UTC"
Jul 30 00:43:37.194883 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:37.194841    2546 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15127h59m46.786360694s"
Jul 30 00:43:37.354585 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:37.354552    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:43:37.354729 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:37.354672    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhf7n" podUID="7368d17a-473f-4725-978f-ee53048b033e"
Jul 30 00:43:37.355444 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:37.355074    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:43:37.355444 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:37.355181    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcwcp" podUID="57b08e9a-8cca-4cde-8f18-e3abe842ac68"
Jul 30 00:43:37.355444 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:37.355243    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:37.355444 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:37.355309    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lfz5" podUID="b7286a2e-d6a2-4db7-912a-3ae863862afa"
Jul 30 00:43:37.385505 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:37.385468    2546 generic.go:356] "Generic (PLEG): container finished" podID="ec5ec4be6f2266139bbde9ed8209c955" containerID="d5e1e120f79d93c90a2ff1fa92cd141b80e8a49a13b998696b417393d5da7e42" exitCode=0
Jul 30 00:43:37.385682 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:37.385647    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal" event={"ID":"ec5ec4be6f2266139bbde9ed8209c955","Type":"ContainerDied","Data":"d5e1e120f79d93c90a2ff1fa92cd141b80e8a49a13b998696b417393d5da7e42"}
Jul 30 00:43:37.394167 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:37.394106    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-st67h" event={"ID":"e0c213ea-4c65-4b0c-813c-0ae9e69047e1","Type":"ContainerStarted","Data":"d909449995472f6c3144516cb4ef7e943adbcc1a2fa4627fe03a06d199366fe1"}
Jul 30 00:43:37.399401 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:37.399260    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bwfnn" event={"ID":"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9","Type":"ContainerStarted","Data":"a732a5dac58e88b56173efb20751e12aab17d88bad9e648238ce2179a1569e76"}
Jul 30 00:43:37.405419 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:37.405393    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-85.ec2.internal" event={"ID":"80e212081188b2e37af81e25e05fed3c","Type":"ContainerStarted","Data":"d6dad9e6ad8536d088f94e87a54daa3b988afc6a68cdb26671bde91d48ceadef"}
Jul 30 00:43:37.420895 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:37.420843    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-85.ec2.internal" podStartSLOduration=3.420823714 podStartE2EDuration="3.420823714s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:37.4204417 +0000 UTC m=+4.643175519" watchObservedRunningTime="2026-07-30 00:43:37.420823714 +0000 UTC m=+4.643557536"
Jul 30 00:43:37.426334 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:37.426306    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz" event={"ID":"49cd90ba-3fc8-4075-83ec-20aa08534f75","Type":"ContainerStarted","Data":"1d0858dc4f339530c789ce3c1cbc8f8f530d6c5c2ca7a27318a1f461a228dab0"}
Jul 30 00:43:37.439232 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:37.439160    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7vhrw" event={"ID":"b429caa1-4e6d-4f0c-b426-1e8b75379b8b","Type":"ContainerStarted","Data":"6fd73898c62c85b10ebabca0598e430c9ba4b2a935edbce063ecd039e9f67c98"}
Jul 30 00:43:37.462223 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:37.462122    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lnhk8" event={"ID":"e419b701-ba27-4810-9851-ae5242cdc2c5","Type":"ContainerStarted","Data":"b06a24e49c8f44ad2366f3ab87b8fb2de3a0a946758625493b6df95872c804d6"}
Jul 30 00:43:37.479578 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:37.479540    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vkxdr" event={"ID":"7eaeb5a9-feed-476d-8a67-083665883cdf","Type":"ContainerStarted","Data":"3ff104a1699ab8a242577f19b54441dae8eb453f06bed8639b086a22c469a9ed"}
Jul 30 00:43:38.504556 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:38.503996    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal" event={"ID":"ec5ec4be6f2266139bbde9ed8209c955","Type":"ContainerStarted","Data":"ea1c2a16b7db41d0c2e5935d955c9147c66ab8c70194a3a57b6641c820fca060"}
Jul 30 00:43:38.792895 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:38.792023    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57b08e9a-8cca-4cde-8f18-e3abe842ac68-metrics-certs\") pod \"network-metrics-daemon-dcwcp\" (UID: \"57b08e9a-8cca-4cde-8f18-e3abe842ac68\") " pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:43:38.792895 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:38.792084    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7286a2e-d6a2-4db7-912a-3ae863862afa-original-pull-secret\") pod \"global-pull-secret-syncer-9lfz5\" (UID: \"b7286a2e-d6a2-4db7-912a-3ae863862afa\") " pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:38.792895 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:38.792225    2546 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 00:43:38.792895 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:38.792291    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7286a2e-d6a2-4db7-912a-3ae863862afa-original-pull-secret podName:b7286a2e-d6a2-4db7-912a-3ae863862afa nodeName:}" failed. No retries permitted until 2026-07-30 00:43:42.792272294 +0000 UTC m=+10.015006108 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b7286a2e-d6a2-4db7-912a-3ae863862afa-original-pull-secret") pod "global-pull-secret-syncer-9lfz5" (UID: "b7286a2e-d6a2-4db7-912a-3ae863862afa") : object "kube-system"/"original-pull-secret" not registered
Jul 30 00:43:38.792895 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:38.792722    2546 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:43:38.792895 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:38.792780    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57b08e9a-8cca-4cde-8f18-e3abe842ac68-metrics-certs podName:57b08e9a-8cca-4cde-8f18-e3abe842ac68 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:42.792761414 +0000 UTC m=+10.015495212 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57b08e9a-8cca-4cde-8f18-e3abe842ac68-metrics-certs") pod "network-metrics-daemon-dcwcp" (UID: "57b08e9a-8cca-4cde-8f18-e3abe842ac68") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:43:38.892685 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:38.892571    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5f68\" (UniqueName: \"kubernetes.io/projected/7368d17a-473f-4725-978f-ee53048b033e-kube-api-access-f5f68\") pod \"network-check-target-zhf7n\" (UID: \"7368d17a-473f-4725-978f-ee53048b033e\") " pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:43:38.892852 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:38.892732    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 00:43:38.892852 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:38.892763    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 00:43:38.892852 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:38.892779    2546 projected.go:196] Error preparing data for projected volume kube-api-access-f5f68 for pod openshift-network-diagnostics/network-check-target-zhf7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:43:38.892852 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:38.892841    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7368d17a-473f-4725-978f-ee53048b033e-kube-api-access-f5f68 podName:7368d17a-473f-4725-978f-ee53048b033e nodeName:}" failed. No retries permitted until 2026-07-30 00:43:42.892821335 +0000 UTC m=+10.115555133 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-f5f68" (UniqueName: "kubernetes.io/projected/7368d17a-473f-4725-978f-ee53048b033e-kube-api-access-f5f68") pod "network-check-target-zhf7n" (UID: "7368d17a-473f-4725-978f-ee53048b033e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:43:39.353993 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:39.353948    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:43:39.354158 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:39.354000    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:39.354158 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:39.354092    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcwcp" podUID="57b08e9a-8cca-4cde-8f18-e3abe842ac68"
Jul 30 00:43:39.354296 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:39.354149    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lfz5" podUID="b7286a2e-d6a2-4db7-912a-3ae863862afa"
Jul 30 00:43:39.354296 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:39.354208    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:43:39.354296 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:39.354273    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhf7n" podUID="7368d17a-473f-4725-978f-ee53048b033e"
Jul 30 00:43:41.351773 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:41.351737    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:43:41.352157 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:41.351787    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:43:41.352157 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:41.351904    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcwcp" podUID="57b08e9a-8cca-4cde-8f18-e3abe842ac68"
Jul 30 00:43:41.352157 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:41.351967    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhf7n" podUID="7368d17a-473f-4725-978f-ee53048b033e"
Jul 30 00:43:41.352157 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:41.351988    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:41.352157 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:41.352082    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lfz5" podUID="b7286a2e-d6a2-4db7-912a-3ae863862afa"
Jul 30 00:43:42.825841 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:42.825811    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57b08e9a-8cca-4cde-8f18-e3abe842ac68-metrics-certs\") pod \"network-metrics-daemon-dcwcp\" (UID: \"57b08e9a-8cca-4cde-8f18-e3abe842ac68\") " pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:43:42.826234 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:42.825917    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7286a2e-d6a2-4db7-912a-3ae863862afa-original-pull-secret\") pod \"global-pull-secret-syncer-9lfz5\" (UID: \"b7286a2e-d6a2-4db7-912a-3ae863862afa\") " pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:42.826234 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:42.825956    2546 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:43:42.826234 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:42.826022    2546 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 00:43:42.826234 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:42.826025    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57b08e9a-8cca-4cde-8f18-e3abe842ac68-metrics-certs podName:57b08e9a-8cca-4cde-8f18-e3abe842ac68 nodeName:}" failed. No retries permitted until 2026-07-30 00:43:50.8260063 +0000 UTC m=+18.048740102 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57b08e9a-8cca-4cde-8f18-e3abe842ac68-metrics-certs") pod "network-metrics-daemon-dcwcp" (UID: "57b08e9a-8cca-4cde-8f18-e3abe842ac68") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:43:42.826234 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:42.826066    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7286a2e-d6a2-4db7-912a-3ae863862afa-original-pull-secret podName:b7286a2e-d6a2-4db7-912a-3ae863862afa nodeName:}" failed. No retries permitted until 2026-07-30 00:43:50.826054904 +0000 UTC m=+18.048788699 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b7286a2e-d6a2-4db7-912a-3ae863862afa-original-pull-secret") pod "global-pull-secret-syncer-9lfz5" (UID: "b7286a2e-d6a2-4db7-912a-3ae863862afa") : object "kube-system"/"original-pull-secret" not registered
Jul 30 00:43:42.926793 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:42.926757    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5f68\" (UniqueName: \"kubernetes.io/projected/7368d17a-473f-4725-978f-ee53048b033e-kube-api-access-f5f68\") pod \"network-check-target-zhf7n\" (UID: \"7368d17a-473f-4725-978f-ee53048b033e\") " pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:43:42.926979 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:42.926958    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 00:43:42.926979 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:42.926978    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 00:43:42.927059 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:42.926990    2546 projected.go:196] Error preparing data for projected volume kube-api-access-f5f68 for pod openshift-network-diagnostics/network-check-target-zhf7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:43:42.927059 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:42.927043    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7368d17a-473f-4725-978f-ee53048b033e-kube-api-access-f5f68 podName:7368d17a-473f-4725-978f-ee53048b033e nodeName:}" failed. No retries permitted until 2026-07-30 00:43:50.927024551 +0000 UTC m=+18.149758352 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-f5f68" (UniqueName: "kubernetes.io/projected/7368d17a-473f-4725-978f-ee53048b033e-kube-api-access-f5f68") pod "network-check-target-zhf7n" (UID: "7368d17a-473f-4725-978f-ee53048b033e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:43:43.353181 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:43.353149    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:43:43.353334 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:43.353253    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcwcp" podUID="57b08e9a-8cca-4cde-8f18-e3abe842ac68"
Jul 30 00:43:43.353797 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:43.353622    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:43.353797 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:43.353640    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:43:43.353797 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:43.353724    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lfz5" podUID="b7286a2e-d6a2-4db7-912a-3ae863862afa"
Jul 30 00:43:43.353797 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:43.353777    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhf7n" podUID="7368d17a-473f-4725-978f-ee53048b033e"
Jul 30 00:43:45.351944 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:45.351908    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:45.352443 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:45.351957    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:43:45.352443 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:45.351910    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:43:45.352443 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:45.352041    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lfz5" podUID="b7286a2e-d6a2-4db7-912a-3ae863862afa"
Jul 30 00:43:45.352443 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:45.352135    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhf7n" podUID="7368d17a-473f-4725-978f-ee53048b033e"
Jul 30 00:43:45.352443 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:45.352219    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcwcp" podUID="57b08e9a-8cca-4cde-8f18-e3abe842ac68"
Jul 30 00:43:47.352339 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:47.352318    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:43:47.352694 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:47.352318    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:43:47.352694 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:47.352428    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcwcp" podUID="57b08e9a-8cca-4cde-8f18-e3abe842ac68"
Jul 30 00:43:47.352694 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:47.352318    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:47.352694 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:47.352506    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhf7n" podUID="7368d17a-473f-4725-978f-ee53048b033e"
Jul 30 00:43:47.352694 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:47.352602    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lfz5" podUID="b7286a2e-d6a2-4db7-912a-3ae863862afa"
Jul 30 00:43:49.352077 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:49.352043    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:49.352077 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:49.352077    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:43:49.352645 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:49.352043    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:43:49.352645 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:49.352178    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lfz5" podUID="b7286a2e-d6a2-4db7-912a-3ae863862afa"
Jul 30 00:43:49.352645 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:49.352254    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcwcp" podUID="57b08e9a-8cca-4cde-8f18-e3abe842ac68"
Jul 30 00:43:49.352645 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:49.352344    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhf7n" podUID="7368d17a-473f-4725-978f-ee53048b033e"
Jul 30 00:43:50.884316 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:50.884286    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57b08e9a-8cca-4cde-8f18-e3abe842ac68-metrics-certs\") pod \"network-metrics-daemon-dcwcp\" (UID: \"57b08e9a-8cca-4cde-8f18-e3abe842ac68\") " pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:43:50.884837 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:50.884323    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7286a2e-d6a2-4db7-912a-3ae863862afa-original-pull-secret\") pod \"global-pull-secret-syncer-9lfz5\" (UID: \"b7286a2e-d6a2-4db7-912a-3ae863862afa\") " pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:50.884837 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:50.884443    2546 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 00:43:50.884837 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:50.884442    2546 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:43:50.884837 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:50.884517    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7286a2e-d6a2-4db7-912a-3ae863862afa-original-pull-secret podName:b7286a2e-d6a2-4db7-912a-3ae863862afa nodeName:}" failed. No retries permitted until 2026-07-30 00:44:06.884498056 +0000 UTC m=+34.107231852 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b7286a2e-d6a2-4db7-912a-3ae863862afa-original-pull-secret") pod "global-pull-secret-syncer-9lfz5" (UID: "b7286a2e-d6a2-4db7-912a-3ae863862afa") : object "kube-system"/"original-pull-secret" not registered
Jul 30 00:43:50.884837 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:50.884629    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57b08e9a-8cca-4cde-8f18-e3abe842ac68-metrics-certs podName:57b08e9a-8cca-4cde-8f18-e3abe842ac68 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:06.884604083 +0000 UTC m=+34.107337897 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57b08e9a-8cca-4cde-8f18-e3abe842ac68-metrics-certs") pod "network-metrics-daemon-dcwcp" (UID: "57b08e9a-8cca-4cde-8f18-e3abe842ac68") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:43:50.985332 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:50.985297    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5f68\" (UniqueName: \"kubernetes.io/projected/7368d17a-473f-4725-978f-ee53048b033e-kube-api-access-f5f68\") pod \"network-check-target-zhf7n\" (UID: \"7368d17a-473f-4725-978f-ee53048b033e\") " pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:43:50.985501 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:50.985459    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 00:43:50.985501 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:50.985475    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 00:43:50.985501 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:50.985486    2546 projected.go:196] Error preparing data for projected volume kube-api-access-f5f68 for pod openshift-network-diagnostics/network-check-target-zhf7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:43:50.985663 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:50.985560    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7368d17a-473f-4725-978f-ee53048b033e-kube-api-access-f5f68 podName:7368d17a-473f-4725-978f-ee53048b033e nodeName:}" failed. No retries permitted until 2026-07-30 00:44:06.985543395 +0000 UTC m=+34.208277211 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-f5f68" (UniqueName: "kubernetes.io/projected/7368d17a-473f-4725-978f-ee53048b033e-kube-api-access-f5f68") pod "network-check-target-zhf7n" (UID: "7368d17a-473f-4725-978f-ee53048b033e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:43:51.352417 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:51.352334    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:43:51.352417 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:51.352387    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:51.352417 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:51.352398    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:43:51.352686 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:51.352469    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lfz5" podUID="b7286a2e-d6a2-4db7-912a-3ae863862afa"
Jul 30 00:43:51.352686 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:51.352575    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhf7n" podUID="7368d17a-473f-4725-978f-ee53048b033e"
Jul 30 00:43:51.352686 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:51.352651    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcwcp" podUID="57b08e9a-8cca-4cde-8f18-e3abe842ac68"
Jul 30 00:43:53.353298 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:53.353175    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:43:53.353797 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:53.353307    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:43:53.353797 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:53.353311    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcwcp" podUID="57b08e9a-8cca-4cde-8f18-e3abe842ac68"
Jul 30 00:43:53.353797 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:53.353419    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhf7n" podUID="7368d17a-473f-4725-978f-ee53048b033e"
Jul 30 00:43:53.353797 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:53.353482    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:53.353797 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:53.353564    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lfz5" podUID="b7286a2e-d6a2-4db7-912a-3ae863862afa"
Jul 30 00:43:55.352990 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:55.352812    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:55.353704 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:55.352815    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:43:55.353704 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:55.353080    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lfz5" podUID="b7286a2e-d6a2-4db7-912a-3ae863862afa"
Jul 30 00:43:55.353704 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:55.352852    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:43:55.353704 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:55.353165    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcwcp" podUID="57b08e9a-8cca-4cde-8f18-e3abe842ac68"
Jul 30 00:43:55.353704 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:55.353252    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhf7n" podUID="7368d17a-473f-4725-978f-ee53048b033e"
Jul 30 00:43:55.537753 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:55.537722    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-st67h" event={"ID":"e0c213ea-4c65-4b0c-813c-0ae9e69047e1","Type":"ContainerStarted","Data":"cdff70182363b5ca9a3e083e48431fadc7364d010a99fd431724b6b93e2bf8b1"}
Jul 30 00:43:55.539256 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:55.539234    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bwfnn" event={"ID":"9fd5e9cd-5cde-452d-b349-31a4c1a12ec9","Type":"ContainerStarted","Data":"6819679b04da63f3e516c948d3b1e97384edb386a05141bd515969ff6d4a443a"}
Jul 30 00:43:55.540765 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:55.540738    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz" event={"ID":"49cd90ba-3fc8-4075-83ec-20aa08534f75","Type":"ContainerStarted","Data":"cac3607aa0cc062f47177183aa0250d2befb99f43e968a0c5d8218c4dc3af3b0"}
Jul 30 00:43:55.542085 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:55.542058    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7vhrw" event={"ID":"b429caa1-4e6d-4f0c-b426-1e8b75379b8b","Type":"ContainerStarted","Data":"a6b91f4bb60d67bf5f818367ece430525f7a939b41f4b7e8d44a12a7730e9117"}
Jul 30 00:43:55.543418 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:55.543396    2546 generic.go:356] "Generic (PLEG): container finished" podID="e419b701-ba27-4810-9851-ae5242cdc2c5" containerID="da79e1b02a6544f9c6bd95a7078b44f5518d353250b993d17895ae34fbaf1650" exitCode=0
Jul 30 00:43:55.543506 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:55.543449    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lnhk8" event={"ID":"e419b701-ba27-4810-9851-ae5242cdc2c5","Type":"ContainerDied","Data":"da79e1b02a6544f9c6bd95a7078b44f5518d353250b993d17895ae34fbaf1650"}
Jul 30 00:43:55.544961 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:55.544943    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vkxdr" event={"ID":"7eaeb5a9-feed-476d-8a67-083665883cdf","Type":"ContainerStarted","Data":"59203d270753509d90431eecbeb8ab9419b0f502b3e0198d9f84a9795a83fecf"}
Jul 30 00:43:55.546417 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:55.546371    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-286nx" event={"ID":"a767f76b-efec-44ec-a0e8-a74c4c589d65","Type":"ContainerStarted","Data":"6bb551b136da6d7b6f6ad987d88ebd1da71c08419158645fe238386d64d81a61"}
Jul 30 00:43:55.548961 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:55.548942    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9" event={"ID":"86d29ffc-9bf4-4a32-a71f-c30674b1924e","Type":"ContainerStarted","Data":"6765f0ba704900a9264c412355c7ffaa225f4f760feb9d86d2851c2d0596d841"}
Jul 30 00:43:55.549038 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:55.548993    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9" event={"ID":"86d29ffc-9bf4-4a32-a71f-c30674b1924e","Type":"ContainerStarted","Data":"9a2b96bad0dc98be1faf3bfd08d7363a6a47a93aa5225e5dbbccae3becacef81"}
Jul 30 00:43:55.549038 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:55.549036    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9" event={"ID":"86d29ffc-9bf4-4a32-a71f-c30674b1924e","Type":"ContainerStarted","Data":"7a1a2107e3ad5b31c599b960e5c986677b914315efb50afad68e24a37eca6cc1"}
Jul 30 00:43:55.549114 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:55.549064    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9" event={"ID":"86d29ffc-9bf4-4a32-a71f-c30674b1924e","Type":"ContainerStarted","Data":"c528d02d5a6dd9baf45d7951ca06477ff09e80839b0a3add6318568d03a78a7b"}
Jul 30 00:43:55.554987 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:55.554953    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal" podStartSLOduration=21.554942325 podStartE2EDuration="21.554942325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:38.521614314 +0000 UTC m=+5.744348142" watchObservedRunningTime="2026-07-30 00:43:55.554942325 +0000 UTC m=+22.777676143"
Jul 30 00:43:55.555282 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:55.555263    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-st67h" podStartSLOduration=3.907445547 podStartE2EDuration="22.555259497s" totalImagesPullingTime="18.64781395s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:55.554692687 +0000 UTC m=+22.777426505" watchObservedRunningTime="2026-07-30 00:43:55.555259497 +0000 UTC m=+22.777993315"
Jul 30 00:43:55.570815 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:55.570780    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vkxdr" podStartSLOduration=3.916609385 podStartE2EDuration="22.570767028s" totalImagesPullingTime="18.654157643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:55.570655704 +0000 UTC m=+22.793389521" watchObservedRunningTime="2026-07-30 00:43:55.570767028 +0000 UTC m=+22.793500846"
Jul 30 00:43:55.610017 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:55.609976    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-286nx" podStartSLOduration=3.932960418 podStartE2EDuration="22.609962767s" totalImagesPullingTime="18.677002349s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:55.609697191 +0000 UTC m=+22.832431011" watchObservedRunningTime="2026-07-30 00:43:55.609962767 +0000 UTC m=+22.832696586"
Jul 30 00:43:55.643811 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:55.643611    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7vhrw" podStartSLOduration=3.994356876 podStartE2EDuration="22.643589874s" totalImagesPullingTime="18.649232998s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:55.62467307 +0000 UTC m=+22.847406889" watchObservedRunningTime="2026-07-30 00:43:55.643589874 +0000 UTC m=+22.866323692"
Jul 30 00:43:56.554875 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:56.554790    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9" event={"ID":"86d29ffc-9bf4-4a32-a71f-c30674b1924e","Type":"ContainerStarted","Data":"0a03c97de94f2eb84fafe1636c03a8f2a5ef66bc8f12012fab78954d45062c57"}
Jul 30 00:43:56.555320 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:56.554910    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9" event={"ID":"86d29ffc-9bf4-4a32-a71f-c30674b1924e","Type":"ContainerStarted","Data":"5214140ba43b6f7ce02a37f26fa232ab0d2a04e8d76bc7abcd9039e6306b1bd4"}
Jul 30 00:43:56.556419 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:56.556385    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fmmn7" event={"ID":"d7aa8886-d7df-47d0-a71a-ad213aab0796","Type":"ContainerStarted","Data":"8d6dcdb3e84c267d0793a313efc8a4dd29387738e8f498178cceeac8410517ab"}
Jul 30 00:43:56.574885 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:56.574832    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bwfnn" podStartSLOduration=4.867390501 podStartE2EDuration="23.574810976s" totalImagesPullingTime="18.707420475s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:55.642688378 +0000 UTC m=+22.865422196" watchObservedRunningTime="2026-07-30 00:43:56.574810976 +0000 UTC m=+23.797544796"
Jul 30 00:43:56.720119 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:56.720079    2546 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 30 00:43:57.230233 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:57.230122    2546 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-30T00:43:56.720103414Z","UUID":"d5608406-684e-472e-8e2a-eb050894de71","Handler":null,"Name":"","Endpoint":""}
Jul 30 00:43:57.231795 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:57.231774    2546 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 30 00:43:57.231905 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:57.231804    2546 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 30 00:43:57.352189 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:57.351930    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:43:57.352189 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:57.351930    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:57.352189 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:57.352079    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:43:57.352189 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:57.352071    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcwcp" podUID="57b08e9a-8cca-4cde-8f18-e3abe842ac68"
Jul 30 00:43:57.352189 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:57.352188    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lfz5" podUID="b7286a2e-d6a2-4db7-912a-3ae863862afa"
Jul 30 00:43:57.352545 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:57.352257    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhf7n" podUID="7368d17a-473f-4725-978f-ee53048b033e"
Jul 30 00:43:57.561663 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:57.561576    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz" event={"ID":"49cd90ba-3fc8-4075-83ec-20aa08534f75","Type":"ContainerStarted","Data":"b55d1b26245d2772c3b2e817c36ad7905cf8505ae61735b5c86abcdf29c458fa"}
Jul 30 00:43:58.295221 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:58.295189    2546 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vkxdr"
Jul 30 00:43:58.296050 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:58.296027    2546 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vkxdr"
Jul 30 00:43:58.310012 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:58.309966    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fmmn7" podStartSLOduration=6.645713866 podStartE2EDuration="25.309947539s" totalImagesPullingTime="18.664233673s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:43:56.574171056 +0000 UTC m=+23.796904875" watchObservedRunningTime="2026-07-30 00:43:58.309947539 +0000 UTC m=+25.532681359"
Jul 30 00:43:58.567420 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:58.567337    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9" event={"ID":"86d29ffc-9bf4-4a32-a71f-c30674b1924e","Type":"ContainerStarted","Data":"8969962e21d35e90c22c4672a6d02f8dab3c15d461178f5f75dc85994c7f5f20"}
Jul 30 00:43:58.568161 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:58.568132    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vkxdr"
Jul 30 00:43:58.568919 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:58.568897    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vkxdr"
Jul 30 00:43:59.351592 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:59.351559    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:43:59.351592 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:59.351583    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:43:59.351831 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:59.351689    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcwcp" podUID="57b08e9a-8cca-4cde-8f18-e3abe842ac68"
Jul 30 00:43:59.351831 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:59.351745    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lfz5" podUID="b7286a2e-d6a2-4db7-912a-3ae863862afa"
Jul 30 00:43:59.351831 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:43:59.351800    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:43:59.351939 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:43:59.351926    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhf7n" podUID="7368d17a-473f-4725-978f-ee53048b033e"
Jul 30 00:44:00.574643 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:00.574602    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9" event={"ID":"86d29ffc-9bf4-4a32-a71f-c30674b1924e","Type":"ContainerStarted","Data":"f9213c1530aa3e0da2f7fde0d28d05e47b7501ea976ab5c289fde28eea8d620c"}
Jul 30 00:44:00.575481 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:00.574854    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:44:00.575481 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:00.574918    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:44:00.575481 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:00.574960    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:44:00.576872 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:00.576848    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz" event={"ID":"49cd90ba-3fc8-4075-83ec-20aa08534f75","Type":"ContainerStarted","Data":"7da9011005b04f3fe31b46c7f008337a06878b070721f4abd14fb0d6d86b5408"}
Jul 30 00:44:00.578842 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:00.578819    2546 generic.go:356] "Generic (PLEG): container finished" podID="e419b701-ba27-4810-9851-ae5242cdc2c5" containerID="bd5ff812f0aab997a0f546c0667fa72c9c2a058a2c840f9f92bd10440d1dcf65" exitCode=0
Jul 30 00:44:00.578941 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:00.578878    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lnhk8" event={"ID":"e419b701-ba27-4810-9851-ae5242cdc2c5","Type":"ContainerDied","Data":"bd5ff812f0aab997a0f546c0667fa72c9c2a058a2c840f9f92bd10440d1dcf65"}
Jul 30 00:44:00.591387 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:00.591363    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:44:00.591835 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:00.591818    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:44:00.600981 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:00.600943    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9" podStartSLOduration=8.876671482999999 podStartE2EDuration="27.600929636s" totalImagesPullingTime="18.724258153s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:00.600668751 +0000 UTC m=+27.823402568" watchObservedRunningTime="2026-07-30 00:44:00.600929636 +0000 UTC m=+27.823663454"
Jul 30 00:44:00.617694 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:00.617652    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6nwz" podStartSLOduration=4.793691377 podStartE2EDuration="27.61763597s" totalImagesPullingTime="22.823944593s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:00.617209131 +0000 UTC m=+27.839942949" watchObservedRunningTime="2026-07-30 00:44:00.61763597 +0000 UTC m=+27.840369831"
Jul 30 00:44:01.355659 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:01.355398    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:44:01.355659 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:01.355399    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:44:01.355905 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:01.355670    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lfz5" podUID="b7286a2e-d6a2-4db7-912a-3ae863862afa"
Jul 30 00:44:01.355905 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:01.355399    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:44:01.355905 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:01.355747    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhf7n" podUID="7368d17a-473f-4725-978f-ee53048b033e"
Jul 30 00:44:01.355905 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:01.355828    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcwcp" podUID="57b08e9a-8cca-4cde-8f18-e3abe842ac68"
Jul 30 00:44:01.922887 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:01.922809    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9lfz5"]
Jul 30 00:44:01.923389 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:01.922915    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:44:01.923389 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:01.923006    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lfz5" podUID="b7286a2e-d6a2-4db7-912a-3ae863862afa"
Jul 30 00:44:01.926165 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:01.926139    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zhf7n"]
Jul 30 00:44:01.926309 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:01.926247    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:44:01.926392 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:01.926351    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhf7n" podUID="7368d17a-473f-4725-978f-ee53048b033e"
Jul 30 00:44:01.926967 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:01.926941    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dcwcp"]
Jul 30 00:44:01.927088 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:01.927074    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:44:01.927214 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:01.927187    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcwcp" podUID="57b08e9a-8cca-4cde-8f18-e3abe842ac68"
Jul 30 00:44:02.586022 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:02.585987    2546 generic.go:356] "Generic (PLEG): container finished" podID="e419b701-ba27-4810-9851-ae5242cdc2c5" containerID="158dd312098fdf5949decae4172bc49bdcdb986de57511fcaa7e5f8782367625" exitCode=0
Jul 30 00:44:02.586266 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:02.586068    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lnhk8" event={"ID":"e419b701-ba27-4810-9851-ae5242cdc2c5","Type":"ContainerDied","Data":"158dd312098fdf5949decae4172bc49bdcdb986de57511fcaa7e5f8782367625"}
Jul 30 00:44:03.352883 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:03.352850    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:44:03.353357 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:03.352952    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lfz5" podUID="b7286a2e-d6a2-4db7-912a-3ae863862afa"
Jul 30 00:44:03.353357 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:03.353021    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:44:03.353357 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:03.353133    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhf7n" podUID="7368d17a-473f-4725-978f-ee53048b033e"
Jul 30 00:44:04.352216 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:04.352049    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:44:04.352389 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:04.352303    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcwcp" podUID="57b08e9a-8cca-4cde-8f18-e3abe842ac68"
Jul 30 00:44:04.592950 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:04.592913    2546 generic.go:356] "Generic (PLEG): container finished" podID="e419b701-ba27-4810-9851-ae5242cdc2c5" containerID="9e0a0fc568bf0b86bace6b5fec211804bd49d2cb6174a902c2ce7ec44f435156" exitCode=0
Jul 30 00:44:04.593396 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:04.592994    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lnhk8" event={"ID":"e419b701-ba27-4810-9851-ae5242cdc2c5","Type":"ContainerDied","Data":"9e0a0fc568bf0b86bace6b5fec211804bd49d2cb6174a902c2ce7ec44f435156"}
Jul 30 00:44:05.352384 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:05.352343    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:44:05.352384 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:05.352382    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:44:05.352650 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:05.352477    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lfz5" podUID="b7286a2e-d6a2-4db7-912a-3ae863862afa"
Jul 30 00:44:05.352915 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:05.352879    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhf7n" podUID="7368d17a-473f-4725-978f-ee53048b033e"
Jul 30 00:44:06.352611 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:06.352577    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:44:06.353126 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:06.352723    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcwcp" podUID="57b08e9a-8cca-4cde-8f18-e3abe842ac68"
Jul 30 00:44:06.908905 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:06.908865    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57b08e9a-8cca-4cde-8f18-e3abe842ac68-metrics-certs\") pod \"network-metrics-daemon-dcwcp\" (UID: \"57b08e9a-8cca-4cde-8f18-e3abe842ac68\") " pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:44:06.909078 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:06.908913    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7286a2e-d6a2-4db7-912a-3ae863862afa-original-pull-secret\") pod \"global-pull-secret-syncer-9lfz5\" (UID: \"b7286a2e-d6a2-4db7-912a-3ae863862afa\") " pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:44:06.909078 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:06.909030    2546 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 00:44:06.909078 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:06.909037    2546 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:44:06.909203 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:06.909090    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7286a2e-d6a2-4db7-912a-3ae863862afa-original-pull-secret podName:b7286a2e-d6a2-4db7-912a-3ae863862afa nodeName:}" failed. No retries permitted until 2026-07-30 00:44:38.90907108 +0000 UTC m=+66.131804887 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b7286a2e-d6a2-4db7-912a-3ae863862afa-original-pull-secret") pod "global-pull-secret-syncer-9lfz5" (UID: "b7286a2e-d6a2-4db7-912a-3ae863862afa") : object "kube-system"/"original-pull-secret" not registered
Jul 30 00:44:06.909203 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:06.909109    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57b08e9a-8cca-4cde-8f18-e3abe842ac68-metrics-certs podName:57b08e9a-8cca-4cde-8f18-e3abe842ac68 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:38.9091 +0000 UTC m=+66.131833795 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57b08e9a-8cca-4cde-8f18-e3abe842ac68-metrics-certs") pod "network-metrics-daemon-dcwcp" (UID: "57b08e9a-8cca-4cde-8f18-e3abe842ac68") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 00:44:07.009839 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:07.009803    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5f68\" (UniqueName: \"kubernetes.io/projected/7368d17a-473f-4725-978f-ee53048b033e-kube-api-access-f5f68\") pod \"network-check-target-zhf7n\" (UID: \"7368d17a-473f-4725-978f-ee53048b033e\") " pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:44:07.010024 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:07.009994    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 00:44:07.010024 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:07.010015    2546 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 00:44:07.010144 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:07.010027    2546 projected.go:196] Error preparing data for projected volume kube-api-access-f5f68 for pod openshift-network-diagnostics/network-check-target-zhf7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:44:07.010144 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:07.010082    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7368d17a-473f-4725-978f-ee53048b033e-kube-api-access-f5f68 podName:7368d17a-473f-4725-978f-ee53048b033e nodeName:}" failed. No retries permitted until 2026-07-30 00:44:39.010068142 +0000 UTC m=+66.232801938 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-f5f68" (UniqueName: "kubernetes.io/projected/7368d17a-473f-4725-978f-ee53048b033e-kube-api-access-f5f68") pod "network-check-target-zhf7n" (UID: "7368d17a-473f-4725-978f-ee53048b033e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 00:44:07.352194 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:07.352112    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:44:07.352358 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:07.352245    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lfz5" podUID="b7286a2e-d6a2-4db7-912a-3ae863862afa"
Jul 30 00:44:07.352358 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:07.352282    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:44:07.352468 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:07.352357    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zhf7n" podUID="7368d17a-473f-4725-978f-ee53048b033e"
Jul 30 00:44:08.352413 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.352378    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:44:08.352964 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:08.352514    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcwcp" podUID="57b08e9a-8cca-4cde-8f18-e3abe842ac68"
Jul 30 00:44:08.547994 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.547969    2546 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeReady"
Jul 30 00:44:08.548103 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.548094    2546 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 30 00:44:08.592643 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.592615    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4xhnp"]
Jul 30 00:44:08.594756 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.594719    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dptvp"]
Jul 30 00:44:08.594886 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.594774    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4xhnp"
Jul 30 00:44:08.596465 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.596441    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dptvp"
Jul 30 00:44:08.597392 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.597160    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 30 00:44:08.599482 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.597447    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 30 00:44:08.599482 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.597571    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-r2q5g\""
Jul 30 00:44:08.599482 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.598834    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mgwl2\""
Jul 30 00:44:08.599482 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.599115    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 30 00:44:08.599482 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.599313    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 30 00:44:08.599816 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.599579    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 30 00:44:08.603918 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.603861    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4xhnp"]
Jul 30 00:44:08.607668 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.607647    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lnhk8" event={"ID":"e419b701-ba27-4810-9851-ae5242cdc2c5","Type":"ContainerStarted","Data":"183b57c10a65d2a22242c2757dbcc75e9f9c22debb8076989174a575cbec2b90"}
Jul 30 00:44:08.608049 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.608015    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dptvp"]
Jul 30 00:44:08.723378 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.723331    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d2b21e5b-4b93-49d1-a49e-51d05d60905f-cert\") pod \"ingress-canary-dptvp\" (UID: \"d2b21e5b-4b93-49d1-a49e-51d05d60905f\") " pod="openshift-ingress-canary/ingress-canary-dptvp"
Jul 30 00:44:08.723378 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.723376    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4d56a868-73af-4d44-946e-13aa14d16532-config-volume\") pod \"dns-default-4xhnp\" (UID: \"4d56a868-73af-4d44-946e-13aa14d16532\") " pod="openshift-dns/dns-default-4xhnp"
Jul 30 00:44:08.723634 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.723398    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dht8d\" (UniqueName: \"kubernetes.io/projected/d2b21e5b-4b93-49d1-a49e-51d05d60905f-kube-api-access-dht8d\") pod \"ingress-canary-dptvp\" (UID: \"d2b21e5b-4b93-49d1-a49e-51d05d60905f\") " pod="openshift-ingress-canary/ingress-canary-dptvp"
Jul 30 00:44:08.723634 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.723414    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d56a868-73af-4d44-946e-13aa14d16532-metrics-tls\") pod \"dns-default-4xhnp\" (UID: \"4d56a868-73af-4d44-946e-13aa14d16532\") " pod="openshift-dns/dns-default-4xhnp"
Jul 30 00:44:08.723634 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.723430    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d56a868-73af-4d44-946e-13aa14d16532-tmp-dir\") pod \"dns-default-4xhnp\" (UID: \"4d56a868-73af-4d44-946e-13aa14d16532\") " pod="openshift-dns/dns-default-4xhnp"
Jul 30 00:44:08.723634 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.723507    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl2h2\" (UniqueName: \"kubernetes.io/projected/4d56a868-73af-4d44-946e-13aa14d16532-kube-api-access-sl2h2\") pod \"dns-default-4xhnp\" (UID: \"4d56a868-73af-4d44-946e-13aa14d16532\") " pod="openshift-dns/dns-default-4xhnp"
Jul 30 00:44:08.824609 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.824566    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d2b21e5b-4b93-49d1-a49e-51d05d60905f-cert\") pod \"ingress-canary-dptvp\" (UID: \"d2b21e5b-4b93-49d1-a49e-51d05d60905f\") " pod="openshift-ingress-canary/ingress-canary-dptvp"
Jul 30 00:44:08.824609 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.824616    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4d56a868-73af-4d44-946e-13aa14d16532-config-volume\") pod \"dns-default-4xhnp\" (UID: \"4d56a868-73af-4d44-946e-13aa14d16532\") " pod="openshift-dns/dns-default-4xhnp"
Jul 30 00:44:08.824826 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.824757    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dht8d\" (UniqueName: \"kubernetes.io/projected/d2b21e5b-4b93-49d1-a49e-51d05d60905f-kube-api-access-dht8d\") pod \"ingress-canary-dptvp\" (UID: \"d2b21e5b-4b93-49d1-a49e-51d05d60905f\") " pod="openshift-ingress-canary/ingress-canary-dptvp"
Jul 30 00:44:08.824826 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.824804    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d56a868-73af-4d44-946e-13aa14d16532-metrics-tls\") pod \"dns-default-4xhnp\" (UID: \"4d56a868-73af-4d44-946e-13aa14d16532\") " pod="openshift-dns/dns-default-4xhnp"
Jul 30 00:44:08.824928 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.824840    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d56a868-73af-4d44-946e-13aa14d16532-tmp-dir\") pod \"dns-default-4xhnp\" (UID: \"4d56a868-73af-4d44-946e-13aa14d16532\") " pod="openshift-dns/dns-default-4xhnp"
Jul 30 00:44:08.824928 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.824873    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl2h2\" (UniqueName: \"kubernetes.io/projected/4d56a868-73af-4d44-946e-13aa14d16532-kube-api-access-sl2h2\") pod \"dns-default-4xhnp\" (UID: \"4d56a868-73af-4d44-946e-13aa14d16532\") " pod="openshift-dns/dns-default-4xhnp"
Jul 30 00:44:08.825026 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:08.824950    2546 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 00:44:08.825026 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:08.825013    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d56a868-73af-4d44-946e-13aa14d16532-metrics-tls podName:4d56a868-73af-4d44-946e-13aa14d16532 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:09.324995952 +0000 UTC m=+36.547729771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4d56a868-73af-4d44-946e-13aa14d16532-metrics-tls") pod "dns-default-4xhnp" (UID: "4d56a868-73af-4d44-946e-13aa14d16532") : secret "dns-default-metrics-tls" not found
Jul 30 00:44:08.825128 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.825114    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d56a868-73af-4d44-946e-13aa14d16532-tmp-dir\") pod \"dns-default-4xhnp\" (UID: \"4d56a868-73af-4d44-946e-13aa14d16532\") " pod="openshift-dns/dns-default-4xhnp"
Jul 30 00:44:08.825176 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.825122    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4d56a868-73af-4d44-946e-13aa14d16532-config-volume\") pod \"dns-default-4xhnp\" (UID: \"4d56a868-73af-4d44-946e-13aa14d16532\") " pod="openshift-dns/dns-default-4xhnp"
Jul 30 00:44:08.828678 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.828647    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d2b21e5b-4b93-49d1-a49e-51d05d60905f-cert\") pod \"ingress-canary-dptvp\" (UID: \"d2b21e5b-4b93-49d1-a49e-51d05d60905f\") " pod="openshift-ingress-canary/ingress-canary-dptvp"
Jul 30 00:44:08.833303 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.833280    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dht8d\" (UniqueName: \"kubernetes.io/projected/d2b21e5b-4b93-49d1-a49e-51d05d60905f-kube-api-access-dht8d\") pod \"ingress-canary-dptvp\" (UID: \"d2b21e5b-4b93-49d1-a49e-51d05d60905f\") " pod="openshift-ingress-canary/ingress-canary-dptvp"
Jul 30 00:44:08.833446 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.833431    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl2h2\" (UniqueName: \"kubernetes.io/projected/4d56a868-73af-4d44-946e-13aa14d16532-kube-api-access-sl2h2\") pod \"dns-default-4xhnp\" (UID: \"4d56a868-73af-4d44-946e-13aa14d16532\") " pod="openshift-dns/dns-default-4xhnp"
Jul 30 00:44:08.915727 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:08.915623    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dptvp"
Jul 30 00:44:09.058036 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:09.058003    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dptvp"]
Jul 30 00:44:09.062056 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:44:09.062029    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2b21e5b_4b93_49d1_a49e_51d05d60905f.slice/crio-c536697d3a39729e464125807a7bf12d6266081b6950c0a2de2d9e06de43454c WatchSource:0}: Error finding container c536697d3a39729e464125807a7bf12d6266081b6950c0a2de2d9e06de43454c: Status 404 returned error can't find the container with id c536697d3a39729e464125807a7bf12d6266081b6950c0a2de2d9e06de43454c
Jul 30 00:44:09.328357 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:09.328328    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d56a868-73af-4d44-946e-13aa14d16532-metrics-tls\") pod \"dns-default-4xhnp\" (UID: \"4d56a868-73af-4d44-946e-13aa14d16532\") " pod="openshift-dns/dns-default-4xhnp"
Jul 30 00:44:09.328517 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:09.328438    2546 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 00:44:09.328517 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:09.328491    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d56a868-73af-4d44-946e-13aa14d16532-metrics-tls podName:4d56a868-73af-4d44-946e-13aa14d16532 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:10.328476677 +0000 UTC m=+37.551210472 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4d56a868-73af-4d44-946e-13aa14d16532-metrics-tls") pod "dns-default-4xhnp" (UID: "4d56a868-73af-4d44-946e-13aa14d16532") : secret "dns-default-metrics-tls" not found
Jul 30 00:44:09.351927 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:09.351893    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:44:09.352004 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:09.351939    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:44:09.354581 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:09.354548    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 00:44:09.355093 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:09.354583    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-r24dv\""
Jul 30 00:44:09.355093 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:09.354605    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 00:44:09.355093 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:09.354760    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 00:44:09.613475 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:09.613386    2546 generic.go:356] "Generic (PLEG): container finished" podID="e419b701-ba27-4810-9851-ae5242cdc2c5" containerID="183b57c10a65d2a22242c2757dbcc75e9f9c22debb8076989174a575cbec2b90" exitCode=0
Jul 30 00:44:09.613475 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:09.613427    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lnhk8" event={"ID":"e419b701-ba27-4810-9851-ae5242cdc2c5","Type":"ContainerDied","Data":"183b57c10a65d2a22242c2757dbcc75e9f9c22debb8076989174a575cbec2b90"}
Jul 30 00:44:09.614748 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:09.614724    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dptvp" event={"ID":"d2b21e5b-4b93-49d1-a49e-51d05d60905f","Type":"ContainerStarted","Data":"c536697d3a39729e464125807a7bf12d6266081b6950c0a2de2d9e06de43454c"}
Jul 30 00:44:10.336458 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:10.336418    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d56a868-73af-4d44-946e-13aa14d16532-metrics-tls\") pod \"dns-default-4xhnp\" (UID: \"4d56a868-73af-4d44-946e-13aa14d16532\") " pod="openshift-dns/dns-default-4xhnp"
Jul 30 00:44:10.336636 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:10.336613    2546 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 00:44:10.336719 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:10.336693    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d56a868-73af-4d44-946e-13aa14d16532-metrics-tls podName:4d56a868-73af-4d44-946e-13aa14d16532 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:12.336670931 +0000 UTC m=+39.559404739 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4d56a868-73af-4d44-946e-13aa14d16532-metrics-tls") pod "dns-default-4xhnp" (UID: "4d56a868-73af-4d44-946e-13aa14d16532") : secret "dns-default-metrics-tls" not found
Jul 30 00:44:10.352476 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:10.352444    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:44:10.355730 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:10.355703    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 00:44:10.356147 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:10.355733    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vdmkk\""
Jul 30 00:44:10.620282 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:10.620197    2546 generic.go:356] "Generic (PLEG): container finished" podID="e419b701-ba27-4810-9851-ae5242cdc2c5" containerID="9a87e7add8aab5ff6351617bc13ae487bf21f022eb6ee411e245093157c8961c" exitCode=0
Jul 30 00:44:10.620282 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:10.620269    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lnhk8" event={"ID":"e419b701-ba27-4810-9851-ae5242cdc2c5","Type":"ContainerDied","Data":"9a87e7add8aab5ff6351617bc13ae487bf21f022eb6ee411e245093157c8961c"}
Jul 30 00:44:11.624616 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:11.624330    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dptvp" event={"ID":"d2b21e5b-4b93-49d1-a49e-51d05d60905f","Type":"ContainerStarted","Data":"0057797f1df7f1754d6e4b57a3b306baf83a50453f80358d004183c5d6600b8a"}
Jul 30 00:44:11.627673 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:11.627648    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lnhk8" event={"ID":"e419b701-ba27-4810-9851-ae5242cdc2c5","Type":"ContainerStarted","Data":"67ca1f160a4e39dc0035f901b9e05a5087f8158d3efca1a688f8c13ce1a6420b"}
Jul 30 00:44:11.639809 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:11.639720    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dptvp" podStartSLOduration=1.423749612 podStartE2EDuration="3.639702589s" totalImagesPullingTime="2.215952977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:11.639698645 +0000 UTC m=+38.862432464" watchObservedRunningTime="2026-07-30 00:44:11.639702589 +0000 UTC m=+38.862436409"
Jul 30 00:44:11.661712 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:11.661665    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-lnhk8" podStartSLOduration=9.620519527 podStartE2EDuration="38.661646989s" totalImagesPullingTime="29.041127462s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:11.660089136 +0000 UTC m=+38.882822953" watchObservedRunningTime="2026-07-30 00:44:11.661646989 +0000 UTC m=+38.884380808"
Jul 30 00:44:12.352283 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:12.352240    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d56a868-73af-4d44-946e-13aa14d16532-metrics-tls\") pod \"dns-default-4xhnp\" (UID: \"4d56a868-73af-4d44-946e-13aa14d16532\") " pod="openshift-dns/dns-default-4xhnp"
Jul 30 00:44:12.352458 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:12.352380    2546 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 00:44:12.352500 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:12.352460    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d56a868-73af-4d44-946e-13aa14d16532-metrics-tls podName:4d56a868-73af-4d44-946e-13aa14d16532 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:16.352443635 +0000 UTC m=+43.575177432 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4d56a868-73af-4d44-946e-13aa14d16532-metrics-tls") pod "dns-default-4xhnp" (UID: "4d56a868-73af-4d44-946e-13aa14d16532") : secret "dns-default-metrics-tls" not found
Jul 30 00:44:13.177156 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.177119    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-4qsk7"]
Jul 30 00:44:13.209429 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.209386    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-4qsk7"]
Jul 30 00:44:13.209429 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.209434    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-58x9b"]
Jul 30 00:44:13.209675 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.209552    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-4qsk7"
Jul 30 00:44:13.212297 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.212277    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 30 00:44:13.212944 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.212923    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-626gl\""
Jul 30 00:44:13.213188 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.213173    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 30 00:44:13.213426 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.213411    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 30 00:44:13.213493 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.213475    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 30 00:44:13.225453 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.225429    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-x82vc"]
Jul 30 00:44:13.225592 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.225559    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-58x9b"
Jul 30 00:44:13.229065 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.229043    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 30 00:44:13.229160 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.229086    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2pjhh\""
Jul 30 00:44:13.229286 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.229268    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 30 00:44:13.241226 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.241203    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-58x9b"]
Jul 30 00:44:13.241334 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.241234    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-x82vc"]
Jul 30 00:44:13.241334 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.241306    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-x82vc"
Jul 30 00:44:13.246601 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.246575    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 30 00:44:13.246716 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.246572    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 30 00:44:13.246716 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.246627    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 30 00:44:13.246716 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.246645    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-8f8xm\""
Jul 30 00:44:13.296165 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.296129    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-lwmtc"]
Jul 30 00:44:13.317495 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.317465    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-8qbqm"]
Jul 30 00:44:13.317670 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.317625    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-lwmtc"
Jul 30 00:44:13.321589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.321563    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-979x9\""
Jul 30 00:44:13.335422 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.335394    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-lwmtc"]
Jul 30 00:44:13.335422 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.335427    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-8qbqm"]
Jul 30 00:44:13.335615 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.335501    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-8qbqm"
Jul 30 00:44:13.338203 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.338181    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 30 00:44:13.338203 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.338193    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 30 00:44:13.338614 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.338597    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 30 00:44:13.338676 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.338637    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-qj2cz\""
Jul 30 00:44:13.338676 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.338651    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 30 00:44:13.343075 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.343059    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 30 00:44:13.359407 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.359379    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3fec0f2e-4a5a-4a55-89ce-4273ee8a5669-serving-cert\") pod \"service-ca-operator-5d6c75f84f-4qsk7\" (UID: \"3fec0f2e-4a5a-4a55-89ce-4273ee8a5669\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-4qsk7"
Jul 30 00:44:13.359407 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.359410    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6eab503e-9794-4b6a-9585-69ac4825aa92-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-x82vc\" (UID: \"6eab503e-9794-4b6a-9585-69ac4825aa92\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-x82vc"
Jul 30 00:44:13.359623 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.359428    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr8c8\" (UniqueName: \"kubernetes.io/projected/6eab503e-9794-4b6a-9585-69ac4825aa92-kube-api-access-dr8c8\") pod \"cluster-samples-operator-6748489656-x82vc\" (UID: \"6eab503e-9794-4b6a-9585-69ac4825aa92\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-x82vc"
Jul 30 00:44:13.359623 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.359512    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9254a1f8-d810-4bea-837c-2698526cb5d4-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-58x9b\" (UID: \"9254a1f8-d810-4bea-837c-2698526cb5d4\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-58x9b"
Jul 30 00:44:13.359623 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.359562    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9254a1f8-d810-4bea-837c-2698526cb5d4-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-58x9b\" (UID: \"9254a1f8-d810-4bea-837c-2698526cb5d4\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-58x9b"
Jul 30 00:44:13.359623 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.359584    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3fec0f2e-4a5a-4a55-89ce-4273ee8a5669-config\") pod \"service-ca-operator-5d6c75f84f-4qsk7\" (UID: \"3fec0f2e-4a5a-4a55-89ce-4273ee8a5669\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-4qsk7"
Jul 30 00:44:13.359816 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.359657    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvr86\" (UniqueName: \"kubernetes.io/projected/3fec0f2e-4a5a-4a55-89ce-4273ee8a5669-kube-api-access-dvr86\") pod \"service-ca-operator-5d6c75f84f-4qsk7\" (UID: \"3fec0f2e-4a5a-4a55-89ce-4273ee8a5669\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-4qsk7"
Jul 30 00:44:13.377592 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.377561    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-wx49h"]
Jul 30 00:44:13.395387 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.395357    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-wx49h"]
Jul 30 00:44:13.395555 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.395469    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-wx49h"
Jul 30 00:44:13.397759 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.397724    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 30 00:44:13.397759 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.397751    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 30 00:44:13.397944 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.397865    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 30 00:44:13.398089 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.398073    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 30 00:44:13.398151 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.398090    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-4gvfm\""
Jul 30 00:44:13.402704 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.402681    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 30 00:44:13.460273 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.460176    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3fec0f2e-4a5a-4a55-89ce-4273ee8a5669-serving-cert\") pod \"service-ca-operator-5d6c75f84f-4qsk7\" (UID: \"3fec0f2e-4a5a-4a55-89ce-4273ee8a5669\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-4qsk7"
Jul 30 00:44:13.460273 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.460215    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6eab503e-9794-4b6a-9585-69ac4825aa92-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-x82vc\" (UID: \"6eab503e-9794-4b6a-9585-69ac4825aa92\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-x82vc"
Jul 30 00:44:13.460273 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.460235    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr8c8\" (UniqueName: \"kubernetes.io/projected/6eab503e-9794-4b6a-9585-69ac4825aa92-kube-api-access-dr8c8\") pod \"cluster-samples-operator-6748489656-x82vc\" (UID: \"6eab503e-9794-4b6a-9585-69ac4825aa92\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-x82vc"
Jul 30 00:44:13.460273 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.460254    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fd5e744-a7db-428d-9d46-c2413213a511-service-ca-bundle\") pod \"insights-operator-5885fdb759-8qbqm\" (UID: \"8fd5e744-a7db-428d-9d46-c2413213a511\") " pod="openshift-insights/insights-operator-5885fdb759-8qbqm"
Jul 30 00:44:13.460594 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.460311    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9254a1f8-d810-4bea-837c-2698526cb5d4-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-58x9b\" (UID: \"9254a1f8-d810-4bea-837c-2698526cb5d4\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-58x9b"
Jul 30 00:44:13.460594 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.460339    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9254a1f8-d810-4bea-837c-2698526cb5d4-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-58x9b\" (UID: \"9254a1f8-d810-4bea-837c-2698526cb5d4\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-58x9b"
Jul 30 00:44:13.460594 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:13.460348    2546 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 30 00:44:13.460594 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.460363    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8fd5e744-a7db-428d-9d46-c2413213a511-tmp\") pod \"insights-operator-5885fdb759-8qbqm\" (UID: \"8fd5e744-a7db-428d-9d46-c2413213a511\") " pod="openshift-insights/insights-operator-5885fdb759-8qbqm"
Jul 30 00:44:13.460594 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.460386    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fd5e744-a7db-428d-9d46-c2413213a511-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-8qbqm\" (UID: \"8fd5e744-a7db-428d-9d46-c2413213a511\") " pod="openshift-insights/insights-operator-5885fdb759-8qbqm"
Jul 30 00:44:13.460594 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.460426    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmwc8\" (UniqueName: \"kubernetes.io/projected/8fd5e744-a7db-428d-9d46-c2413213a511-kube-api-access-rmwc8\") pod \"insights-operator-5885fdb759-8qbqm\" (UID: \"8fd5e744-a7db-428d-9d46-c2413213a511\") " pod="openshift-insights/insights-operator-5885fdb759-8qbqm"
Jul 30 00:44:13.460594 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:13.460451    2546 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 00:44:13.460594 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:13.460483    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eab503e-9794-4b6a-9585-69ac4825aa92-samples-operator-tls podName:6eab503e-9794-4b6a-9585-69ac4825aa92 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:13.9604616 +0000 UTC m=+41.183195414 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6eab503e-9794-4b6a-9585-69ac4825aa92-samples-operator-tls") pod "cluster-samples-operator-6748489656-x82vc" (UID: "6eab503e-9794-4b6a-9585-69ac4825aa92") : secret "samples-operator-tls" not found
Jul 30 00:44:13.460594 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:13.460505    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9254a1f8-d810-4bea-837c-2698526cb5d4-networking-console-plugin-cert podName:9254a1f8-d810-4bea-837c-2698526cb5d4 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:13.960492726 +0000 UTC m=+41.183226528 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9254a1f8-d810-4bea-837c-2698526cb5d4-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-58x9b" (UID: "9254a1f8-d810-4bea-837c-2698526cb5d4") : secret "networking-console-plugin-cert" not found
Jul 30 00:44:13.460594 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.460550    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8fd5e744-a7db-428d-9d46-c2413213a511-snapshots\") pod \"insights-operator-5885fdb759-8qbqm\" (UID: \"8fd5e744-a7db-428d-9d46-c2413213a511\") " pod="openshift-insights/insights-operator-5885fdb759-8qbqm"
Jul 30 00:44:13.460594 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.460582    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3fec0f2e-4a5a-4a55-89ce-4273ee8a5669-config\") pod \"service-ca-operator-5d6c75f84f-4qsk7\" (UID: \"3fec0f2e-4a5a-4a55-89ce-4273ee8a5669\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-4qsk7"
Jul 30 00:44:13.461079 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.460604    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8fd5e744-a7db-428d-9d46-c2413213a511-serving-cert\") pod \"insights-operator-5885fdb759-8qbqm\" (UID: \"8fd5e744-a7db-428d-9d46-c2413213a511\") " pod="openshift-insights/insights-operator-5885fdb759-8qbqm"
Jul 30 00:44:13.461079 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.460672    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7kkx\" (UniqueName: \"kubernetes.io/projected/74f03256-e2e1-49a7-ba01-80f096ba44e3-kube-api-access-c7kkx\") pod \"network-check-source-7d88cb55d8-lwmtc\" (UID: \"74f03256-e2e1-49a7-ba01-80f096ba44e3\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-lwmtc"
Jul 30 00:44:13.461079 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.460712    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvr86\" (UniqueName: \"kubernetes.io/projected/3fec0f2e-4a5a-4a55-89ce-4273ee8a5669-kube-api-access-dvr86\") pod \"service-ca-operator-5d6c75f84f-4qsk7\" (UID: \"3fec0f2e-4a5a-4a55-89ce-4273ee8a5669\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-4qsk7"
Jul 30 00:44:13.461079 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.461062    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3fec0f2e-4a5a-4a55-89ce-4273ee8a5669-config\") pod \"service-ca-operator-5d6c75f84f-4qsk7\" (UID: \"3fec0f2e-4a5a-4a55-89ce-4273ee8a5669\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-4qsk7"
Jul 30 00:44:13.461199 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.461158    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9254a1f8-d810-4bea-837c-2698526cb5d4-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-58x9b\" (UID: \"9254a1f8-d810-4bea-837c-2698526cb5d4\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-58x9b"
Jul 30 00:44:13.464451 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.464432    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3fec0f2e-4a5a-4a55-89ce-4273ee8a5669-serving-cert\") pod \"service-ca-operator-5d6c75f84f-4qsk7\" (UID: \"3fec0f2e-4a5a-4a55-89ce-4273ee8a5669\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-4qsk7"
Jul 30 00:44:13.471076 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.471045    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvr86\" (UniqueName: \"kubernetes.io/projected/3fec0f2e-4a5a-4a55-89ce-4273ee8a5669-kube-api-access-dvr86\") pod \"service-ca-operator-5d6c75f84f-4qsk7\" (UID: \"3fec0f2e-4a5a-4a55-89ce-4273ee8a5669\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-4qsk7"
Jul 30 00:44:13.471191 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.471168    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr8c8\" (UniqueName: \"kubernetes.io/projected/6eab503e-9794-4b6a-9585-69ac4825aa92-kube-api-access-dr8c8\") pod \"cluster-samples-operator-6748489656-x82vc\" (UID: \"6eab503e-9794-4b6a-9585-69ac4825aa92\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-x82vc"
Jul 30 00:44:13.520639 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.520604    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-4qsk7"
Jul 30 00:44:13.561589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.561558    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7kkx\" (UniqueName: \"kubernetes.io/projected/74f03256-e2e1-49a7-ba01-80f096ba44e3-kube-api-access-c7kkx\") pod \"network-check-source-7d88cb55d8-lwmtc\" (UID: \"74f03256-e2e1-49a7-ba01-80f096ba44e3\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-lwmtc"
Jul 30 00:44:13.561748 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.561599    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c56ae285-bf61-4f11-b846-827ea2e7594e-serving-cert\") pod \"console-operator-584dcb6b54-wx49h\" (UID: \"c56ae285-bf61-4f11-b846-827ea2e7594e\") " pod="openshift-console-operator/console-operator-584dcb6b54-wx49h"
Jul 30 00:44:13.561821 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.561774    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fd5e744-a7db-428d-9d46-c2413213a511-service-ca-bundle\") pod \"insights-operator-5885fdb759-8qbqm\" (UID: \"8fd5e744-a7db-428d-9d46-c2413213a511\") " pod="openshift-insights/insights-operator-5885fdb759-8qbqm"
Jul 30 00:44:13.561903 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.561883    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8fd5e744-a7db-428d-9d46-c2413213a511-tmp\") pod \"insights-operator-5885fdb759-8qbqm\" (UID: \"8fd5e744-a7db-428d-9d46-c2413213a511\") " pod="openshift-insights/insights-operator-5885fdb759-8qbqm"
Jul 30 00:44:13.562013 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.561918    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fd5e744-a7db-428d-9d46-c2413213a511-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-8qbqm\" (UID: \"8fd5e744-a7db-428d-9d46-c2413213a511\") " pod="openshift-insights/insights-operator-5885fdb759-8qbqm"
Jul 30 00:44:13.562013 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.561945    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmwc8\" (UniqueName: \"kubernetes.io/projected/8fd5e744-a7db-428d-9d46-c2413213a511-kube-api-access-rmwc8\") pod \"insights-operator-5885fdb759-8qbqm\" (UID: \"8fd5e744-a7db-428d-9d46-c2413213a511\") " pod="openshift-insights/insights-operator-5885fdb759-8qbqm"
Jul 30 00:44:13.562013 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.561981    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c56ae285-bf61-4f11-b846-827ea2e7594e-config\") pod \"console-operator-584dcb6b54-wx49h\" (UID: \"c56ae285-bf61-4f11-b846-827ea2e7594e\") " pod="openshift-console-operator/console-operator-584dcb6b54-wx49h"
Jul 30 00:44:13.562013 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.562011    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8fd5e744-a7db-428d-9d46-c2413213a511-snapshots\") pod \"insights-operator-5885fdb759-8qbqm\" (UID: \"8fd5e744-a7db-428d-9d46-c2413213a511\") " pod="openshift-insights/insights-operator-5885fdb759-8qbqm"
Jul 30 00:44:13.562243 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.562040    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7259f\" (UniqueName: \"kubernetes.io/projected/c56ae285-bf61-4f11-b846-827ea2e7594e-kube-api-access-7259f\") pod \"console-operator-584dcb6b54-wx49h\" (UID: \"c56ae285-bf61-4f11-b846-827ea2e7594e\") " pod="openshift-console-operator/console-operator-584dcb6b54-wx49h"
Jul 30 00:44:13.562243 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.562077    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8fd5e744-a7db-428d-9d46-c2413213a511-serving-cert\") pod \"insights-operator-5885fdb759-8qbqm\" (UID: \"8fd5e744-a7db-428d-9d46-c2413213a511\") " pod="openshift-insights/insights-operator-5885fdb759-8qbqm"
Jul 30 00:44:13.562243 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.562121    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c56ae285-bf61-4f11-b846-827ea2e7594e-trusted-ca\") pod \"console-operator-584dcb6b54-wx49h\" (UID: \"c56ae285-bf61-4f11-b846-827ea2e7594e\") " pod="openshift-console-operator/console-operator-584dcb6b54-wx49h"
Jul 30 00:44:13.562439 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.562416    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fd5e744-a7db-428d-9d46-c2413213a511-service-ca-bundle\") pod \"insights-operator-5885fdb759-8qbqm\" (UID: \"8fd5e744-a7db-428d-9d46-c2413213a511\") " pod="openshift-insights/insights-operator-5885fdb759-8qbqm"
Jul 30 00:44:13.562489 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.562475    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8fd5e744-a7db-428d-9d46-c2413213a511-tmp\") pod \"insights-operator-5885fdb759-8qbqm\" (UID: \"8fd5e744-a7db-428d-9d46-c2413213a511\") " pod="openshift-insights/insights-operator-5885fdb759-8qbqm"
Jul 30 00:44:13.562686 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.562666    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8fd5e744-a7db-428d-9d46-c2413213a511-snapshots\") pod \"insights-operator-5885fdb759-8qbqm\" (UID: \"8fd5e744-a7db-428d-9d46-c2413213a511\") " pod="openshift-insights/insights-operator-5885fdb759-8qbqm"
Jul 30 00:44:13.563099 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.563078    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fd5e744-a7db-428d-9d46-c2413213a511-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-8qbqm\" (UID: \"8fd5e744-a7db-428d-9d46-c2413213a511\") " pod="openshift-insights/insights-operator-5885fdb759-8qbqm"
Jul 30 00:44:13.564631 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.564602    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8fd5e744-a7db-428d-9d46-c2413213a511-serving-cert\") pod \"insights-operator-5885fdb759-8qbqm\" (UID: \"8fd5e744-a7db-428d-9d46-c2413213a511\") " pod="openshift-insights/insights-operator-5885fdb759-8qbqm"
Jul 30 00:44:13.571490 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.571465    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7kkx\" (UniqueName: \"kubernetes.io/projected/74f03256-e2e1-49a7-ba01-80f096ba44e3-kube-api-access-c7kkx\") pod \"network-check-source-7d88cb55d8-lwmtc\" (UID: \"74f03256-e2e1-49a7-ba01-80f096ba44e3\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-lwmtc"
Jul 30 00:44:13.571590 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.571468    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmwc8\" (UniqueName: \"kubernetes.io/projected/8fd5e744-a7db-428d-9d46-c2413213a511-kube-api-access-rmwc8\") pod \"insights-operator-5885fdb759-8qbqm\" (UID: \"8fd5e744-a7db-428d-9d46-c2413213a511\") " pod="openshift-insights/insights-operator-5885fdb759-8qbqm"
Jul 30 00:44:13.626977 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.626949    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-lwmtc"
Jul 30 00:44:13.645902 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.645874    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-8qbqm"
Jul 30 00:44:13.659149 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.659116    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-4qsk7"]
Jul 30 00:44:13.662927 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.662901    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c56ae285-bf61-4f11-b846-827ea2e7594e-config\") pod \"console-operator-584dcb6b54-wx49h\" (UID: \"c56ae285-bf61-4f11-b846-827ea2e7594e\") " pod="openshift-console-operator/console-operator-584dcb6b54-wx49h"
Jul 30 00:44:13.663041 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.662941    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7259f\" (UniqueName: \"kubernetes.io/projected/c56ae285-bf61-4f11-b846-827ea2e7594e-kube-api-access-7259f\") pod \"console-operator-584dcb6b54-wx49h\" (UID: \"c56ae285-bf61-4f11-b846-827ea2e7594e\") " pod="openshift-console-operator/console-operator-584dcb6b54-wx49h"
Jul 30 00:44:13.663041 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.662979    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c56ae285-bf61-4f11-b846-827ea2e7594e-trusted-ca\") pod \"console-operator-584dcb6b54-wx49h\" (UID: \"c56ae285-bf61-4f11-b846-827ea2e7594e\") " pod="openshift-console-operator/console-operator-584dcb6b54-wx49h"
Jul 30 00:44:13.663041 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.663011    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c56ae285-bf61-4f11-b846-827ea2e7594e-serving-cert\") pod \"console-operator-584dcb6b54-wx49h\" (UID: \"c56ae285-bf61-4f11-b846-827ea2e7594e\") " pod="openshift-console-operator/console-operator-584dcb6b54-wx49h"
Jul 30 00:44:13.663679 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.663651    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c56ae285-bf61-4f11-b846-827ea2e7594e-config\") pod \"console-operator-584dcb6b54-wx49h\" (UID: \"c56ae285-bf61-4f11-b846-827ea2e7594e\") " pod="openshift-console-operator/console-operator-584dcb6b54-wx49h"
Jul 30 00:44:13.663840 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:44:13.663815    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fec0f2e_4a5a_4a55_89ce_4273ee8a5669.slice/crio-a7373797ccfd3bba588a233afab768cfffc1c191e70f74901a197a83b8445321 WatchSource:0}: Error finding container a7373797ccfd3bba588a233afab768cfffc1c191e70f74901a197a83b8445321: Status 404 returned error can't find the container with id a7373797ccfd3bba588a233afab768cfffc1c191e70f74901a197a83b8445321
Jul 30 00:44:13.664535 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.664496    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c56ae285-bf61-4f11-b846-827ea2e7594e-trusted-ca\") pod \"console-operator-584dcb6b54-wx49h\" (UID: \"c56ae285-bf61-4f11-b846-827ea2e7594e\") " pod="openshift-console-operator/console-operator-584dcb6b54-wx49h"
Jul 30 00:44:13.665856 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.665834    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c56ae285-bf61-4f11-b846-827ea2e7594e-serving-cert\") pod \"console-operator-584dcb6b54-wx49h\" (UID: \"c56ae285-bf61-4f11-b846-827ea2e7594e\") " pod="openshift-console-operator/console-operator-584dcb6b54-wx49h"
Jul 30 00:44:13.670230 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.670210    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7259f\" (UniqueName: \"kubernetes.io/projected/c56ae285-bf61-4f11-b846-827ea2e7594e-kube-api-access-7259f\") pod \"console-operator-584dcb6b54-wx49h\" (UID: \"c56ae285-bf61-4f11-b846-827ea2e7594e\") " pod="openshift-console-operator/console-operator-584dcb6b54-wx49h"
Jul 30 00:44:13.705362 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.705323    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-wx49h"
Jul 30 00:44:13.800944 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.800254    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-lwmtc"]
Jul 30 00:44:13.805000 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:44:13.804954    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74f03256_e2e1_49a7_ba01_80f096ba44e3.slice/crio-48de612176034c5b7abdbe1484eb8fb26ac3dd2732e6521737c38e6b1b63b9e6 WatchSource:0}: Error finding container 48de612176034c5b7abdbe1484eb8fb26ac3dd2732e6521737c38e6b1b63b9e6: Status 404 returned error can't find the container with id 48de612176034c5b7abdbe1484eb8fb26ac3dd2732e6521737c38e6b1b63b9e6
Jul 30 00:44:13.825731 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.825702    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-8qbqm"]
Jul 30 00:44:13.828936 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:44:13.828904    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fd5e744_a7db_428d_9d46_c2413213a511.slice/crio-30b1e7b1e6ea403fd10f9e50742d803ac6420dd25401fcdbf25c4b40f373f332 WatchSource:0}: Error finding container 30b1e7b1e6ea403fd10f9e50742d803ac6420dd25401fcdbf25c4b40f373f332: Status 404 returned error can't find the container with id 30b1e7b1e6ea403fd10f9e50742d803ac6420dd25401fcdbf25c4b40f373f332
Jul 30 00:44:13.869420 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.869387    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-wx49h"]
Jul 30 00:44:13.872738 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:44:13.872706    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc56ae285_bf61_4f11_b846_827ea2e7594e.slice/crio-f8e511ba1764d1ba10e0b7c5df4afd8a71a4f78df772e255cbd7bc9a4f16b3aa WatchSource:0}: Error finding container f8e511ba1764d1ba10e0b7c5df4afd8a71a4f78df772e255cbd7bc9a4f16b3aa: Status 404 returned error can't find the container with id f8e511ba1764d1ba10e0b7c5df4afd8a71a4f78df772e255cbd7bc9a4f16b3aa
Jul 30 00:44:13.967014 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.966981    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9254a1f8-d810-4bea-837c-2698526cb5d4-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-58x9b\" (UID: \"9254a1f8-d810-4bea-837c-2698526cb5d4\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-58x9b"
Jul 30 00:44:13.967154 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:13.967064    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6eab503e-9794-4b6a-9585-69ac4825aa92-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-x82vc\" (UID: \"6eab503e-9794-4b6a-9585-69ac4825aa92\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-x82vc"
Jul 30 00:44:13.967189 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:13.967147    2546 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 00:44:13.967189 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:13.967164    2546 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 30 00:44:13.967264 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:13.967219    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eab503e-9794-4b6a-9585-69ac4825aa92-samples-operator-tls podName:6eab503e-9794-4b6a-9585-69ac4825aa92 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:14.967203308 +0000 UTC m=+42.189937106 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6eab503e-9794-4b6a-9585-69ac4825aa92-samples-operator-tls") pod "cluster-samples-operator-6748489656-x82vc" (UID: "6eab503e-9794-4b6a-9585-69ac4825aa92") : secret "samples-operator-tls" not found
Jul 30 00:44:13.967264 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:13.967233    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9254a1f8-d810-4bea-837c-2698526cb5d4-networking-console-plugin-cert podName:9254a1f8-d810-4bea-837c-2698526cb5d4 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:14.96722687 +0000 UTC m=+42.189960666 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9254a1f8-d810-4bea-837c-2698526cb5d4-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-58x9b" (UID: "9254a1f8-d810-4bea-837c-2698526cb5d4") : secret "networking-console-plugin-cert" not found
Jul 30 00:44:14.637398 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:14.637323    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-4qsk7" event={"ID":"3fec0f2e-4a5a-4a55-89ce-4273ee8a5669","Type":"ContainerStarted","Data":"a7373797ccfd3bba588a233afab768cfffc1c191e70f74901a197a83b8445321"}
Jul 30 00:44:14.639081 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:14.639054    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-wx49h" event={"ID":"c56ae285-bf61-4f11-b846-827ea2e7594e","Type":"ContainerStarted","Data":"f8e511ba1764d1ba10e0b7c5df4afd8a71a4f78df772e255cbd7bc9a4f16b3aa"}
Jul 30 00:44:14.641374 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:14.641348    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-lwmtc" event={"ID":"74f03256-e2e1-49a7-ba01-80f096ba44e3","Type":"ContainerStarted","Data":"48de612176034c5b7abdbe1484eb8fb26ac3dd2732e6521737c38e6b1b63b9e6"}
Jul 30 00:44:14.643879 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:14.643844    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-8qbqm" event={"ID":"8fd5e744-a7db-428d-9d46-c2413213a511","Type":"ContainerStarted","Data":"30b1e7b1e6ea403fd10f9e50742d803ac6420dd25401fcdbf25c4b40f373f332"}
Jul 30 00:44:14.976108 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:14.976067    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6eab503e-9794-4b6a-9585-69ac4825aa92-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-x82vc\" (UID: \"6eab503e-9794-4b6a-9585-69ac4825aa92\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-x82vc"
Jul 30 00:44:14.976308 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:14.976160    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9254a1f8-d810-4bea-837c-2698526cb5d4-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-58x9b\" (UID: \"9254a1f8-d810-4bea-837c-2698526cb5d4\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-58x9b"
Jul 30 00:44:14.976308 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:14.976290    2546 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 00:44:14.976424 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:14.976355    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9254a1f8-d810-4bea-837c-2698526cb5d4-networking-console-plugin-cert podName:9254a1f8-d810-4bea-837c-2698526cb5d4 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:16.976336574 +0000 UTC m=+44.199070373 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9254a1f8-d810-4bea-837c-2698526cb5d4-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-58x9b" (UID: "9254a1f8-d810-4bea-837c-2698526cb5d4") : secret "networking-console-plugin-cert" not found
Jul 30 00:44:14.976826 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:14.976807    2546 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 30 00:44:14.976896 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:14.976867    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eab503e-9794-4b6a-9585-69ac4825aa92-samples-operator-tls podName:6eab503e-9794-4b6a-9585-69ac4825aa92 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:16.976849564 +0000 UTC m=+44.199583363 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6eab503e-9794-4b6a-9585-69ac4825aa92-samples-operator-tls") pod "cluster-samples-operator-6748489656-x82vc" (UID: "6eab503e-9794-4b6a-9585-69ac4825aa92") : secret "samples-operator-tls" not found
Jul 30 00:44:16.390535 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:16.390492    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d56a868-73af-4d44-946e-13aa14d16532-metrics-tls\") pod \"dns-default-4xhnp\" (UID: \"4d56a868-73af-4d44-946e-13aa14d16532\") " pod="openshift-dns/dns-default-4xhnp"
Jul 30 00:44:16.390933 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:16.390675    2546 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 00:44:16.390933 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:16.390745    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d56a868-73af-4d44-946e-13aa14d16532-metrics-tls podName:4d56a868-73af-4d44-946e-13aa14d16532 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:24.39072884 +0000 UTC m=+51.613462636 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4d56a868-73af-4d44-946e-13aa14d16532-metrics-tls") pod "dns-default-4xhnp" (UID: "4d56a868-73af-4d44-946e-13aa14d16532") : secret "dns-default-metrics-tls" not found
Jul 30 00:44:16.994379 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:16.994337    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6eab503e-9794-4b6a-9585-69ac4825aa92-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-x82vc\" (UID: \"6eab503e-9794-4b6a-9585-69ac4825aa92\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-x82vc"
Jul 30 00:44:16.994516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:16.994429    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9254a1f8-d810-4bea-837c-2698526cb5d4-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-58x9b\" (UID: \"9254a1f8-d810-4bea-837c-2698526cb5d4\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-58x9b"
Jul 30 00:44:16.994595 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:16.994576    2546 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 00:44:16.994650 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:16.994638    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9254a1f8-d810-4bea-837c-2698526cb5d4-networking-console-plugin-cert podName:9254a1f8-d810-4bea-837c-2698526cb5d4 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:20.994623954 +0000 UTC m=+48.217357763 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9254a1f8-d810-4bea-837c-2698526cb5d4-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-58x9b" (UID: "9254a1f8-d810-4bea-837c-2698526cb5d4") : secret "networking-console-plugin-cert" not found
Jul 30 00:44:16.994729 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:16.994576    2546 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 30 00:44:16.994729 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:16.994705    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eab503e-9794-4b6a-9585-69ac4825aa92-samples-operator-tls podName:6eab503e-9794-4b6a-9585-69ac4825aa92 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:20.994692924 +0000 UTC m=+48.217426727 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6eab503e-9794-4b6a-9585-69ac4825aa92-samples-operator-tls") pod "cluster-samples-operator-6748489656-x82vc" (UID: "6eab503e-9794-4b6a-9585-69ac4825aa92") : secret "samples-operator-tls" not found
Jul 30 00:44:18.640325 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:18.640293    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-x85f9"]
Jul 30 00:44:18.662852 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:18.662819    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-x85f9"]
Jul 30 00:44:18.663014 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:18.662942    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-x85f9"
Jul 30 00:44:18.665607 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:18.665579    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 30 00:44:18.665767 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:18.665581    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 30 00:44:18.665767 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:18.665648    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-2c642\""
Jul 30 00:44:18.810377 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:18.810334    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzc7f\" (UniqueName: \"kubernetes.io/projected/24e359fe-816f-4977-a4e6-3d3bb82a1634-kube-api-access-zzc7f\") pod \"migrator-6865c9967-x85f9\" (UID: \"24e359fe-816f-4977-a4e6-3d3bb82a1634\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-x85f9"
Jul 30 00:44:18.911542 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:18.911451    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzc7f\" (UniqueName: \"kubernetes.io/projected/24e359fe-816f-4977-a4e6-3d3bb82a1634-kube-api-access-zzc7f\") pod \"migrator-6865c9967-x85f9\" (UID: \"24e359fe-816f-4977-a4e6-3d3bb82a1634\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-x85f9"
Jul 30 00:44:18.920070 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:18.920040    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzc7f\" (UniqueName: \"kubernetes.io/projected/24e359fe-816f-4977-a4e6-3d3bb82a1634-kube-api-access-zzc7f\") pod \"migrator-6865c9967-x85f9\" (UID: \"24e359fe-816f-4977-a4e6-3d3bb82a1634\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-x85f9"
Jul 30 00:44:18.971454 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:18.971419    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-x85f9"
Jul 30 00:44:19.229467 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:19.229441    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-x85f9"]
Jul 30 00:44:19.235718 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:44:19.235692    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24e359fe_816f_4977_a4e6_3d3bb82a1634.slice/crio-04492099c6cca59c267ee8cf52d19b08baeb34869df843c97021838bcc9af258 WatchSource:0}: Error finding container 04492099c6cca59c267ee8cf52d19b08baeb34869df843c97021838bcc9af258: Status 404 returned error can't find the container with id 04492099c6cca59c267ee8cf52d19b08baeb34869df843c97021838bcc9af258
Jul 30 00:44:19.661212 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:19.661174    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-4qsk7" event={"ID":"3fec0f2e-4a5a-4a55-89ce-4273ee8a5669","Type":"ContainerStarted","Data":"5320258ad0c7dd191411548e99785c3b8ac8a99b8eeda1be0117a0f4f90fee4a"}
Jul 30 00:44:19.667834 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:19.667808    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/0.log"
Jul 30 00:44:19.668006 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:19.667854    2546 generic.go:356] "Generic (PLEG): container finished" podID="c56ae285-bf61-4f11-b846-827ea2e7594e" containerID="2f07f99ac4bb4547c4ddd2b91f5cf9fc014ce666b8def02e46a63ec280b61b24" exitCode=255
Jul 30 00:44:19.668006 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:19.667907    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-wx49h" event={"ID":"c56ae285-bf61-4f11-b846-827ea2e7594e","Type":"ContainerDied","Data":"2f07f99ac4bb4547c4ddd2b91f5cf9fc014ce666b8def02e46a63ec280b61b24"}
Jul 30 00:44:19.668460 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:19.668441    2546 scope.go:117] "RemoveContainer" containerID="2f07f99ac4bb4547c4ddd2b91f5cf9fc014ce666b8def02e46a63ec280b61b24"
Jul 30 00:44:19.669860 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:19.669833    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-lwmtc" event={"ID":"74f03256-e2e1-49a7-ba01-80f096ba44e3","Type":"ContainerStarted","Data":"af3f3c953bc848638f4838780ec82a226cdd11caaa36e732aad78d72d2d0ca75"}
Jul 30 00:44:19.671085 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:19.671064    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-x85f9" event={"ID":"24e359fe-816f-4977-a4e6-3d3bb82a1634","Type":"ContainerStarted","Data":"04492099c6cca59c267ee8cf52d19b08baeb34869df843c97021838bcc9af258"}
Jul 30 00:44:19.672654 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:19.672629    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-8qbqm" event={"ID":"8fd5e744-a7db-428d-9d46-c2413213a511","Type":"ContainerStarted","Data":"73335d160264ac29ab155b73d69c5c60a3dd6ab27c33d8ffe93c8a9a33b8e95b"}
Jul 30 00:44:19.680875 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:19.680837    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-4qsk7" podStartSLOduration=1.245389214 podStartE2EDuration="6.680823955s" totalImagesPullingTime="5.435434741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:19.679692923 +0000 UTC m=+46.902426766" watchObservedRunningTime="2026-07-30 00:44:19.680823955 +0000 UTC m=+46.903557773"
Jul 30 00:44:19.696059 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:19.696007    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-lwmtc" podStartSLOduration=1.4004423369999999 podStartE2EDuration="6.695992336s" totalImagesPullingTime="5.295549999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:19.694887265 +0000 UTC m=+46.917621085" watchObservedRunningTime="2026-07-30 00:44:19.695992336 +0000 UTC m=+46.918726155"
Jul 30 00:44:19.730063 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:19.729134    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-8qbqm" podStartSLOduration=1.454374487 podStartE2EDuration="6.729111868s" totalImagesPullingTime="5.274737381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:19.728435018 +0000 UTC m=+46.951168833" watchObservedRunningTime="2026-07-30 00:44:19.729111868 +0000 UTC m=+46.951845688"
Jul 30 00:44:20.679576 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:20.679515    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/1.log"
Jul 30 00:44:20.680046 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:20.679995    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/0.log"
Jul 30 00:44:20.680046 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:20.680027    2546 generic.go:356] "Generic (PLEG): container finished" podID="c56ae285-bf61-4f11-b846-827ea2e7594e" containerID="dd37f28cdc93037ed75e7f19e2a90c46128abd8c6c41e0896e4c2a2bcff2ef21" exitCode=255
Jul 30 00:44:20.680166 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:20.680123    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-wx49h" event={"ID":"c56ae285-bf61-4f11-b846-827ea2e7594e","Type":"ContainerDied","Data":"dd37f28cdc93037ed75e7f19e2a90c46128abd8c6c41e0896e4c2a2bcff2ef21"}
Jul 30 00:44:20.680226 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:20.680175    2546 scope.go:117] "RemoveContainer" containerID="2f07f99ac4bb4547c4ddd2b91f5cf9fc014ce666b8def02e46a63ec280b61b24"
Jul 30 00:44:20.680483 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:20.680464    2546 scope.go:117] "RemoveContainer" containerID="dd37f28cdc93037ed75e7f19e2a90c46128abd8c6c41e0896e4c2a2bcff2ef21"
Jul 30 00:44:20.681052 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:20.680674    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-wx49h_openshift-console-operator(c56ae285-bf61-4f11-b846-827ea2e7594e)\"" pod="openshift-console-operator/console-operator-584dcb6b54-wx49h" podUID="c56ae285-bf61-4f11-b846-827ea2e7594e"
Jul 30 00:44:21.031119 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:21.031084    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6eab503e-9794-4b6a-9585-69ac4825aa92-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-x82vc\" (UID: \"6eab503e-9794-4b6a-9585-69ac4825aa92\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-x82vc"
Jul 30 00:44:21.031319 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:21.031254    2546 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 30 00:44:21.031319 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:21.031279    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9254a1f8-d810-4bea-837c-2698526cb5d4-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-58x9b\" (UID: \"9254a1f8-d810-4bea-837c-2698526cb5d4\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-58x9b"
Jul 30 00:44:21.031410 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:21.031333    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eab503e-9794-4b6a-9585-69ac4825aa92-samples-operator-tls podName:6eab503e-9794-4b6a-9585-69ac4825aa92 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:29.031312552 +0000 UTC m=+56.254046368 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6eab503e-9794-4b6a-9585-69ac4825aa92-samples-operator-tls") pod "cluster-samples-operator-6748489656-x82vc" (UID: "6eab503e-9794-4b6a-9585-69ac4825aa92") : secret "samples-operator-tls" not found
Jul 30 00:44:21.031410 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:21.031387    2546 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 00:44:21.031506 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:21.031452    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9254a1f8-d810-4bea-837c-2698526cb5d4-networking-console-plugin-cert podName:9254a1f8-d810-4bea-837c-2698526cb5d4 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:29.031434796 +0000 UTC m=+56.254168594 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9254a1f8-d810-4bea-837c-2698526cb5d4-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-58x9b" (UID: "9254a1f8-d810-4bea-837c-2698526cb5d4") : secret "networking-console-plugin-cert" not found
Jul 30 00:44:21.325473 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:21.325398    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jhdjl"]
Jul 30 00:44:21.347346 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:21.347309    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jhdjl"]
Jul 30 00:44:21.347506 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:21.347427    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:21.349680 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:21.349661    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-p9rd8\""
Jul 30 00:44:21.349799 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:21.349703    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 30 00:44:21.350717 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:21.350701    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 30 00:44:21.534644 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:21.534613    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42841174-db24-4585-a998-29845763f1e4-crio-socket\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:21.534644 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:21.534647    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snfgd\" (UniqueName: \"kubernetes.io/projected/42841174-db24-4585-a998-29845763f1e4-kube-api-access-snfgd\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:21.534895 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:21.534695    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42841174-db24-4585-a998-29845763f1e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:21.537772 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:21.535221    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42841174-db24-4585-a998-29845763f1e4-data-volume\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:21.537772 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:21.535292    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42841174-db24-4585-a998-29845763f1e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:21.635954 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:21.635923    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42841174-db24-4585-a998-29845763f1e4-crio-socket\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:21.636117 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:21.635969    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snfgd\" (UniqueName: \"kubernetes.io/projected/42841174-db24-4585-a998-29845763f1e4-kube-api-access-snfgd\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:21.636117 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:21.636110    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42841174-db24-4585-a998-29845763f1e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:21.636234 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:21.636136    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42841174-db24-4585-a998-29845763f1e4-crio-socket\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:21.636234 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:21.636191    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42841174-db24-4585-a998-29845763f1e4-data-volume\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:21.636234 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:21.636212    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42841174-db24-4585-a998-29845763f1e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:21.636371 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:21.636242    2546 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 00:44:21.636371 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:21.636307    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42841174-db24-4585-a998-29845763f1e4-insights-runtime-extractor-tls podName:42841174-db24-4585-a998-29845763f1e4 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:22.136283389 +0000 UTC m=+49.359017200 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42841174-db24-4585-a998-29845763f1e4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jhdjl" (UID: "42841174-db24-4585-a998-29845763f1e4") : secret "insights-runtime-extractor-tls" not found
Jul 30 00:44:21.636371 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:21.636361    2546 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 00:44:21.636486 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:21.636436    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42841174-db24-4585-a998-29845763f1e4-kube-rbac-proxy-cm podName:42841174-db24-4585-a998-29845763f1e4 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:22.136419857 +0000 UTC m=+49.359153657 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42841174-db24-4585-a998-29845763f1e4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jhdjl" (UID: "42841174-db24-4585-a998-29845763f1e4") : configmap "kube-rbac-proxy" not found
Jul 30 00:44:21.636575 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:21.636557    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42841174-db24-4585-a998-29845763f1e4-data-volume\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:21.644611 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:21.644591    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snfgd\" (UniqueName: \"kubernetes.io/projected/42841174-db24-4585-a998-29845763f1e4-kube-api-access-snfgd\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:21.684057 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:21.684033    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/1.log"
Jul 30 00:44:21.684439 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:21.684352    2546 scope.go:117] "RemoveContainer" containerID="dd37f28cdc93037ed75e7f19e2a90c46128abd8c6c41e0896e4c2a2bcff2ef21"
Jul 30 00:44:21.684801 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:21.684545    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-wx49h_openshift-console-operator(c56ae285-bf61-4f11-b846-827ea2e7594e)\"" pod="openshift-console-operator/console-operator-584dcb6b54-wx49h" podUID="c56ae285-bf61-4f11-b846-827ea2e7594e"
Jul 30 00:44:21.686840 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:21.686077    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-x85f9" event={"ID":"24e359fe-816f-4977-a4e6-3d3bb82a1634","Type":"ContainerStarted","Data":"665f570f5130acc0c306c96f05ce1e8ef5bb0245c13e60926bbdeb105921506b"}
Jul 30 00:44:22.141276 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:22.141237    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42841174-db24-4585-a998-29845763f1e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:22.141471 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:22.141300    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42841174-db24-4585-a998-29845763f1e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:22.141471 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:22.141391    2546 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 00:44:22.141471 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:22.141392    2546 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 00:44:22.141471 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:22.141442    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42841174-db24-4585-a998-29845763f1e4-kube-rbac-proxy-cm podName:42841174-db24-4585-a998-29845763f1e4 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:23.141428879 +0000 UTC m=+50.364162675 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42841174-db24-4585-a998-29845763f1e4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jhdjl" (UID: "42841174-db24-4585-a998-29845763f1e4") : configmap "kube-rbac-proxy" not found
Jul 30 00:44:22.141471 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:22.141454    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42841174-db24-4585-a998-29845763f1e4-insights-runtime-extractor-tls podName:42841174-db24-4585-a998-29845763f1e4 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:23.141449007 +0000 UTC m=+50.364182802 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42841174-db24-4585-a998-29845763f1e4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jhdjl" (UID: "42841174-db24-4585-a998-29845763f1e4") : secret "insights-runtime-extractor-tls" not found
Jul 30 00:44:22.690556 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:22.690500    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-x85f9" event={"ID":"24e359fe-816f-4977-a4e6-3d3bb82a1634","Type":"ContainerStarted","Data":"447224b78b19e27caaa88aff85ea8320b4eac8eb2f8f752ed1535f398bac3a41"}
Jul 30 00:44:22.705509 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:22.705459    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-x85f9" podStartSLOduration=2.3875402709999998 podStartE2EDuration="4.705440932s" totalImagesPullingTime="2.317900661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:22.704360628 +0000 UTC m=+49.927094448" watchObservedRunningTime="2026-07-30 00:44:22.705440932 +0000 UTC m=+49.928174750"
Jul 30 00:44:23.148421 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:23.148387    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42841174-db24-4585-a998-29845763f1e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:23.148655 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:23.148445    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42841174-db24-4585-a998-29845763f1e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:23.148655 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:23.148508    2546 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 00:44:23.148655 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:23.148580    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42841174-db24-4585-a998-29845763f1e4-kube-rbac-proxy-cm podName:42841174-db24-4585-a998-29845763f1e4 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:25.148566214 +0000 UTC m=+52.371300010 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42841174-db24-4585-a998-29845763f1e4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jhdjl" (UID: "42841174-db24-4585-a998-29845763f1e4") : configmap "kube-rbac-proxy" not found
Jul 30 00:44:23.148655 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:23.148581    2546 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 00:44:23.148655 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:23.148643    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42841174-db24-4585-a998-29845763f1e4-insights-runtime-extractor-tls podName:42841174-db24-4585-a998-29845763f1e4 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:25.148627057 +0000 UTC m=+52.371360869 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42841174-db24-4585-a998-29845763f1e4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jhdjl" (UID: "42841174-db24-4585-a998-29845763f1e4") : secret "insights-runtime-extractor-tls" not found
Jul 30 00:44:23.706371 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:23.706338    2546 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-wx49h"
Jul 30 00:44:23.706851 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:23.706393    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-wx49h"
Jul 30 00:44:23.706851 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:23.706784    2546 scope.go:117] "RemoveContainer" containerID="dd37f28cdc93037ed75e7f19e2a90c46128abd8c6c41e0896e4c2a2bcff2ef21"
Jul 30 00:44:23.706975 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:23.706951    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-wx49h_openshift-console-operator(c56ae285-bf61-4f11-b846-827ea2e7594e)\"" pod="openshift-console-operator/console-operator-584dcb6b54-wx49h" podUID="c56ae285-bf61-4f11-b846-827ea2e7594e"
Jul 30 00:44:24.307564 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:24.307538    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-st67h_e0c213ea-4c65-4b0c-813c-0ae9e69047e1/dns-node-resolver/0.log"
Jul 30 00:44:24.458983 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:24.458942    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d56a868-73af-4d44-946e-13aa14d16532-metrics-tls\") pod \"dns-default-4xhnp\" (UID: \"4d56a868-73af-4d44-946e-13aa14d16532\") " pod="openshift-dns/dns-default-4xhnp"
Jul 30 00:44:24.459156 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:24.459091    2546 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 00:44:24.459156 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:24.459151    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d56a868-73af-4d44-946e-13aa14d16532-metrics-tls podName:4d56a868-73af-4d44-946e-13aa14d16532 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:40.459135919 +0000 UTC m=+67.681869715 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4d56a868-73af-4d44-946e-13aa14d16532-metrics-tls") pod "dns-default-4xhnp" (UID: "4d56a868-73af-4d44-946e-13aa14d16532") : secret "dns-default-metrics-tls" not found
Jul 30 00:44:24.904747 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:24.904717    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7vhrw_b429caa1-4e6d-4f0c-b426-1e8b75379b8b/node-ca/0.log"
Jul 30 00:44:25.165335 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:25.165248    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42841174-db24-4585-a998-29845763f1e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:25.165335 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:25.165310    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42841174-db24-4585-a998-29845763f1e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:25.165541 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:25.165370    2546 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 00:44:25.165541 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:25.165402    2546 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 00:44:25.165541 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:25.165421    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42841174-db24-4585-a998-29845763f1e4-kube-rbac-proxy-cm podName:42841174-db24-4585-a998-29845763f1e4 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:29.165406123 +0000 UTC m=+56.388139920 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42841174-db24-4585-a998-29845763f1e4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jhdjl" (UID: "42841174-db24-4585-a998-29845763f1e4") : configmap "kube-rbac-proxy" not found
Jul 30 00:44:25.165541 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:25.165463    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42841174-db24-4585-a998-29845763f1e4-insights-runtime-extractor-tls podName:42841174-db24-4585-a998-29845763f1e4 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:29.165444883 +0000 UTC m=+56.388178688 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42841174-db24-4585-a998-29845763f1e4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jhdjl" (UID: "42841174-db24-4585-a998-29845763f1e4") : secret "insights-runtime-extractor-tls" not found
Jul 30 00:44:25.904247 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:25.904220    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dptvp_d2b21e5b-4b93-49d1-a49e-51d05d60905f/serve-healthcheck-canary/0.log"
Jul 30 00:44:26.303765 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:26.303735    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-x85f9_24e359fe-816f-4977-a4e6-3d3bb82a1634/migrator/0.log"
Jul 30 00:44:26.504630 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:26.504600    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-x85f9_24e359fe-816f-4977-a4e6-3d3bb82a1634/graceful-termination/0.log"
Jul 30 00:44:29.096597 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:29.096564    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9254a1f8-d810-4bea-837c-2698526cb5d4-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-58x9b\" (UID: \"9254a1f8-d810-4bea-837c-2698526cb5d4\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-58x9b"
Jul 30 00:44:29.096969 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:29.096638    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6eab503e-9794-4b6a-9585-69ac4825aa92-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-x82vc\" (UID: \"6eab503e-9794-4b6a-9585-69ac4825aa92\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-x82vc"
Jul 30 00:44:29.096969 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:29.096720    2546 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 00:44:29.096969 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:29.096789    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9254a1f8-d810-4bea-837c-2698526cb5d4-networking-console-plugin-cert podName:9254a1f8-d810-4bea-837c-2698526cb5d4 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:45.09677199 +0000 UTC m=+72.319505786 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9254a1f8-d810-4bea-837c-2698526cb5d4-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-58x9b" (UID: "9254a1f8-d810-4bea-837c-2698526cb5d4") : secret "networking-console-plugin-cert" not found
Jul 30 00:44:29.098897 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:29.098879    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6eab503e-9794-4b6a-9585-69ac4825aa92-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-x82vc\" (UID: \"6eab503e-9794-4b6a-9585-69ac4825aa92\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-x82vc"
Jul 30 00:44:29.149860 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:29.149831    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-x82vc"
Jul 30 00:44:29.197319 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:29.197263    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42841174-db24-4585-a998-29845763f1e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:29.197465 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:29.197418    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42841174-db24-4585-a998-29845763f1e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:29.197465 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:29.197442    2546 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 00:44:29.197679 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:29.197509    2546 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 00:44:29.197679 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:29.197541    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42841174-db24-4585-a998-29845763f1e4-insights-runtime-extractor-tls podName:42841174-db24-4585-a998-29845763f1e4 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:37.197501768 +0000 UTC m=+64.420235605 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42841174-db24-4585-a998-29845763f1e4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jhdjl" (UID: "42841174-db24-4585-a998-29845763f1e4") : secret "insights-runtime-extractor-tls" not found
Jul 30 00:44:29.197679 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:29.197584    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42841174-db24-4585-a998-29845763f1e4-kube-rbac-proxy-cm podName:42841174-db24-4585-a998-29845763f1e4 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:37.197566062 +0000 UTC m=+64.420299900 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42841174-db24-4585-a998-29845763f1e4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jhdjl" (UID: "42841174-db24-4585-a998-29845763f1e4") : configmap "kube-rbac-proxy" not found
Jul 30 00:44:29.270334 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:29.270299    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-x82vc"]
Jul 30 00:44:29.712182 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:29.712144    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-x82vc" event={"ID":"6eab503e-9794-4b6a-9585-69ac4825aa92","Type":"ContainerStarted","Data":"c6802852c8693ba8f69f8c312a698e7ebae6b3a79b4dbace357e095686fa4ccd"}
Jul 30 00:44:31.720664 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:31.720626    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-x82vc" event={"ID":"6eab503e-9794-4b6a-9585-69ac4825aa92","Type":"ContainerStarted","Data":"37c163f11d8acdfc55f34502b797ea30a3424dd2ed011ced9e8dca5409a98871"}
Jul 30 00:44:31.721043 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:31.720683    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-x82vc" event={"ID":"6eab503e-9794-4b6a-9585-69ac4825aa92","Type":"ContainerStarted","Data":"9ce9ccffdfac1602d0a2be7d50e381b981655b3c134ad1e8e3aec3f99f471677"}
Jul 30 00:44:31.736616 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:31.736568    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-x82vc" podStartSLOduration=16.851875635 podStartE2EDuration="18.736552392s" totalImagesPullingTime="1.884676757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:31.735914765 +0000 UTC m=+58.958648582" watchObservedRunningTime="2026-07-30 00:44:31.736552392 +0000 UTC m=+58.959286209"
Jul 30 00:44:32.606694 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:32.606662    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fnvt9"
Jul 30 00:44:37.262483 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:37.262450    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42841174-db24-4585-a998-29845763f1e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:37.262870 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:37.262558    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42841174-db24-4585-a998-29845763f1e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:37.262870 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:37.262662    2546 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 00:44:37.262870 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:37.262714    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42841174-db24-4585-a998-29845763f1e4-insights-runtime-extractor-tls podName:42841174-db24-4585-a998-29845763f1e4 nodeName:}" failed. No retries permitted until 2026-07-30 00:44:53.262700198 +0000 UTC m=+80.485433994 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42841174-db24-4585-a998-29845763f1e4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jhdjl" (UID: "42841174-db24-4585-a998-29845763f1e4") : secret "insights-runtime-extractor-tls" not found
Jul 30 00:44:37.262976 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:37.262946    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42841174-db24-4585-a998-29845763f1e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:37.352758 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:37.352730    2546 scope.go:117] "RemoveContainer" containerID="dd37f28cdc93037ed75e7f19e2a90c46128abd8c6c41e0896e4c2a2bcff2ef21"
Jul 30 00:44:37.739874 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:37.739842    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 00:44:37.740232 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:37.740216    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/1.log"
Jul 30 00:44:37.740282 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:37.740254    2546 generic.go:356] "Generic (PLEG): container finished" podID="c56ae285-bf61-4f11-b846-827ea2e7594e" containerID="d4180a0ba92c18b1d0f8cdd3a83d32fa68b2a44dd12bf6c05551cee7642c3c07" exitCode=255
Jul 30 00:44:37.740327 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:37.740286    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-wx49h" event={"ID":"c56ae285-bf61-4f11-b846-827ea2e7594e","Type":"ContainerDied","Data":"d4180a0ba92c18b1d0f8cdd3a83d32fa68b2a44dd12bf6c05551cee7642c3c07"}
Jul 30 00:44:37.740365 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:37.740334    2546 scope.go:117] "RemoveContainer" containerID="dd37f28cdc93037ed75e7f19e2a90c46128abd8c6c41e0896e4c2a2bcff2ef21"
Jul 30 00:44:37.740728 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:37.740710    2546 scope.go:117] "RemoveContainer" containerID="d4180a0ba92c18b1d0f8cdd3a83d32fa68b2a44dd12bf6c05551cee7642c3c07"
Jul 30 00:44:37.740954 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:37.740934    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-wx49h_openshift-console-operator(c56ae285-bf61-4f11-b846-827ea2e7594e)\"" pod="openshift-console-operator/console-operator-584dcb6b54-wx49h" podUID="c56ae285-bf61-4f11-b846-827ea2e7594e"
Jul 30 00:44:38.748952 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:38.748926    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 00:44:38.976972 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:38.976922    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57b08e9a-8cca-4cde-8f18-e3abe842ac68-metrics-certs\") pod \"network-metrics-daemon-dcwcp\" (UID: \"57b08e9a-8cca-4cde-8f18-e3abe842ac68\") " pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:44:38.976972 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:38.976970    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7286a2e-d6a2-4db7-912a-3ae863862afa-original-pull-secret\") pod \"global-pull-secret-syncer-9lfz5\" (UID: \"b7286a2e-d6a2-4db7-912a-3ae863862afa\") " pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:44:38.979631 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:38.979614    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 00:44:38.979679 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:38.979614    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 00:44:38.989788 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:38.989761    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7286a2e-d6a2-4db7-912a-3ae863862afa-original-pull-secret\") pod \"global-pull-secret-syncer-9lfz5\" (UID: \"b7286a2e-d6a2-4db7-912a-3ae863862afa\") " pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:44:38.989892 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:38.989825    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57b08e9a-8cca-4cde-8f18-e3abe842ac68-metrics-certs\") pod \"network-metrics-daemon-dcwcp\" (UID: \"57b08e9a-8cca-4cde-8f18-e3abe842ac68\") " pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:44:39.063318 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:39.063250    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lfz5"
Jul 30 00:44:39.078007 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:39.077983    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5f68\" (UniqueName: \"kubernetes.io/projected/7368d17a-473f-4725-978f-ee53048b033e-kube-api-access-f5f68\") pod \"network-check-target-zhf7n\" (UID: \"7368d17a-473f-4725-978f-ee53048b033e\") " pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:44:39.080315 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:39.080295    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5f68\" (UniqueName: \"kubernetes.io/projected/7368d17a-473f-4725-978f-ee53048b033e-kube-api-access-f5f68\") pod \"network-check-target-zhf7n\" (UID: \"7368d17a-473f-4725-978f-ee53048b033e\") " pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:44:39.164458 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:39.164433    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vdmkk\""
Jul 30 00:44:39.172516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:39.172492    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcwcp"
Jul 30 00:44:39.180183 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:39.180161    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9lfz5"]
Jul 30 00:44:39.182988 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:44:39.182964    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7286a2e_d6a2_4db7_912a_3ae863862afa.slice/crio-46c5039da1f8bd380232644e43b4a67039db0533c5f8ce2caff06fb9ae737aaf WatchSource:0}: Error finding container 46c5039da1f8bd380232644e43b4a67039db0533c5f8ce2caff06fb9ae737aaf: Status 404 returned error can't find the container with id 46c5039da1f8bd380232644e43b4a67039db0533c5f8ce2caff06fb9ae737aaf
Jul 30 00:44:39.291220 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:39.291192    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dcwcp"]
Jul 30 00:44:39.294293 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:44:39.294266    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57b08e9a_8cca_4cde_8f18_e3abe842ac68.slice/crio-b877def6a338d290b6beb01cd836e9e68e8c64db7afaf91254fdf04f61031f8f WatchSource:0}: Error finding container b877def6a338d290b6beb01cd836e9e68e8c64db7afaf91254fdf04f61031f8f: Status 404 returned error can't find the container with id b877def6a338d290b6beb01cd836e9e68e8c64db7afaf91254fdf04f61031f8f
Jul 30 00:44:39.370291 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:39.370219    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-r24dv\""
Jul 30 00:44:39.378202 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:39.378172    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:44:39.498848 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:39.498818    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zhf7n"]
Jul 30 00:44:39.501940 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:44:39.501914    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7368d17a_473f_4725_978f_ee53048b033e.slice/crio-893112ff0a6ca44876c3f26eaafa9079050aa892830e8c3f3c93c8897909d4e4 WatchSource:0}: Error finding container 893112ff0a6ca44876c3f26eaafa9079050aa892830e8c3f3c93c8897909d4e4: Status 404 returned error can't find the container with id 893112ff0a6ca44876c3f26eaafa9079050aa892830e8c3f3c93c8897909d4e4
Jul 30 00:44:39.754275 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:39.754239    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zhf7n" event={"ID":"7368d17a-473f-4725-978f-ee53048b033e","Type":"ContainerStarted","Data":"b162128b46d5308298550053e87184fffc7ba6a4ff2b1978d3a2a1126b9e9454"}
Jul 30 00:44:39.754773 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:39.754300    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zhf7n" event={"ID":"7368d17a-473f-4725-978f-ee53048b033e","Type":"ContainerStarted","Data":"893112ff0a6ca44876c3f26eaafa9079050aa892830e8c3f3c93c8897909d4e4"}
Jul 30 00:44:39.754773 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:39.754320    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:44:39.755502 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:39.755476    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dcwcp" event={"ID":"57b08e9a-8cca-4cde-8f18-e3abe842ac68","Type":"ContainerStarted","Data":"b877def6a338d290b6beb01cd836e9e68e8c64db7afaf91254fdf04f61031f8f"}
Jul 30 00:44:39.756479 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:39.756458    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9lfz5" event={"ID":"b7286a2e-d6a2-4db7-912a-3ae863862afa","Type":"ContainerStarted","Data":"46c5039da1f8bd380232644e43b4a67039db0533c5f8ce2caff06fb9ae737aaf"}
Jul 30 00:44:39.770500 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:39.770456    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zhf7n" podStartSLOduration=66.770443381 podStartE2EDuration="1m6.770443381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:43:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:39.769695611 +0000 UTC m=+66.992429449" watchObservedRunningTime="2026-07-30 00:44:39.770443381 +0000 UTC m=+66.993177177"
Jul 30 00:44:40.490871 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:40.490834    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d56a868-73af-4d44-946e-13aa14d16532-metrics-tls\") pod \"dns-default-4xhnp\" (UID: \"4d56a868-73af-4d44-946e-13aa14d16532\") " pod="openshift-dns/dns-default-4xhnp"
Jul 30 00:44:40.493547 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:40.493511    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d56a868-73af-4d44-946e-13aa14d16532-metrics-tls\") pod \"dns-default-4xhnp\" (UID: \"4d56a868-73af-4d44-946e-13aa14d16532\") " pod="openshift-dns/dns-default-4xhnp"
Jul 30 00:44:40.713326 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:40.713109    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-r2q5g\""
Jul 30 00:44:40.721447 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:40.721255    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4xhnp"
Jul 30 00:44:40.952581 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:40.952531    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4xhnp"]
Jul 30 00:44:40.955628 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:44:40.955601    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d56a868_73af_4d44_946e_13aa14d16532.slice/crio-a8c23c3bf65bf6a79b0d1921c1ac28fc85b91b39093bd0cbfddb128f2ab153c9 WatchSource:0}: Error finding container a8c23c3bf65bf6a79b0d1921c1ac28fc85b91b39093bd0cbfddb128f2ab153c9: Status 404 returned error can't find the container with id a8c23c3bf65bf6a79b0d1921c1ac28fc85b91b39093bd0cbfddb128f2ab153c9
Jul 30 00:44:41.766048 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:41.766007    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dcwcp" event={"ID":"57b08e9a-8cca-4cde-8f18-e3abe842ac68","Type":"ContainerStarted","Data":"2b65ca1450026e839ffe21ecd00e6fcb75e3aa636906769c12739e6cb47e72a8"}
Jul 30 00:44:41.766221 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:41.766075    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dcwcp" event={"ID":"57b08e9a-8cca-4cde-8f18-e3abe842ac68","Type":"ContainerStarted","Data":"e78fba15dbf9912c143ddb142fafc3cf7dc33684fe4b87dbf68872331cca5355"}
Jul 30 00:44:41.767216 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:41.767185    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4xhnp" event={"ID":"4d56a868-73af-4d44-946e-13aa14d16532","Type":"ContainerStarted","Data":"a8c23c3bf65bf6a79b0d1921c1ac28fc85b91b39093bd0cbfddb128f2ab153c9"}
Jul 30 00:44:41.782992 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:41.782936    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dcwcp" podStartSLOduration=66.933778723 podStartE2EDuration="1m8.78292062s" totalImagesPullingTime="1.849141897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:41.781982993 +0000 UTC m=+69.004716810" watchObservedRunningTime="2026-07-30 00:44:41.78292062 +0000 UTC m=+69.005654437"
Jul 30 00:44:43.706438 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:43.706400    2546 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-wx49h"
Jul 30 00:44:43.706837 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:43.706469    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-wx49h"
Jul 30 00:44:43.706888 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:43.706842    2546 scope.go:117] "RemoveContainer" containerID="d4180a0ba92c18b1d0f8cdd3a83d32fa68b2a44dd12bf6c05551cee7642c3c07"
Jul 30 00:44:43.707039 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:43.707013    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-wx49h_openshift-console-operator(c56ae285-bf61-4f11-b846-827ea2e7594e)\"" pod="openshift-console-operator/console-operator-584dcb6b54-wx49h" podUID="c56ae285-bf61-4f11-b846-827ea2e7594e"
Jul 30 00:44:43.777053 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:43.777010    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9lfz5" event={"ID":"b7286a2e-d6a2-4db7-912a-3ae863862afa","Type":"ContainerStarted","Data":"36a5e5d9a063a09aafa841a32100902a74d754a9374279b906cee4dc4845c384"}
Jul 30 00:44:43.791813 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:43.791759    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9lfz5" podStartSLOduration=66.980007448 podStartE2EDuration="1m10.791743843s" totalImagesPullingTime="3.811736395s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:43.790733047 +0000 UTC m=+71.013466867" watchObservedRunningTime="2026-07-30 00:44:43.791743843 +0000 UTC m=+71.014477639"
Jul 30 00:44:44.123361 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.123329    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw"]
Jul 30 00:44:44.125731 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.125715    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw"
Jul 30 00:44:44.128204 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.128179    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 30 00:44:44.128442 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.128411    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 30 00:44:44.128541 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.128451    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 30 00:44:44.129381 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.129361    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 30 00:44:44.129482 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.129426    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 30 00:44:44.129482 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.129460    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 30 00:44:44.129482 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.129472    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 30 00:44:44.139272 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.139251    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw"]
Jul 30 00:44:44.221427 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.221398    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0a948f44-c233-4ef6-b8e6-83d27ef6bc4f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f9d955c7c-j87fw\" (UID: \"0a948f44-c233-4ef6-b8e6-83d27ef6bc4f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw"
Jul 30 00:44:44.221553 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.221486    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtzvd\" (UniqueName: \"kubernetes.io/projected/0a948f44-c233-4ef6-b8e6-83d27ef6bc4f-kube-api-access-rtzvd\") pod \"cluster-proxy-proxy-agent-f9d955c7c-j87fw\" (UID: \"0a948f44-c233-4ef6-b8e6-83d27ef6bc4f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw"
Jul 30 00:44:44.221621 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.221548    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a948f44-c233-4ef6-b8e6-83d27ef6bc4f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f9d955c7c-j87fw\" (UID: \"0a948f44-c233-4ef6-b8e6-83d27ef6bc4f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw"
Jul 30 00:44:44.221681 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.221615    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0a948f44-c233-4ef6-b8e6-83d27ef6bc4f-hub\") pod \"cluster-proxy-proxy-agent-f9d955c7c-j87fw\" (UID: \"0a948f44-c233-4ef6-b8e6-83d27ef6bc4f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw"
Jul 30 00:44:44.221681 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.221658    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0a948f44-c233-4ef6-b8e6-83d27ef6bc4f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f9d955c7c-j87fw\" (UID: \"0a948f44-c233-4ef6-b8e6-83d27ef6bc4f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw"
Jul 30 00:44:44.221784 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.221715    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0a948f44-c233-4ef6-b8e6-83d27ef6bc4f-ca\") pod \"cluster-proxy-proxy-agent-f9d955c7c-j87fw\" (UID: \"0a948f44-c233-4ef6-b8e6-83d27ef6bc4f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw"
Jul 30 00:44:44.322818 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.322786    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0a948f44-c233-4ef6-b8e6-83d27ef6bc4f-ca\") pod \"cluster-proxy-proxy-agent-f9d955c7c-j87fw\" (UID: \"0a948f44-c233-4ef6-b8e6-83d27ef6bc4f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw"
Jul 30 00:44:44.322948 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.322827    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0a948f44-c233-4ef6-b8e6-83d27ef6bc4f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f9d955c7c-j87fw\" (UID: \"0a948f44-c233-4ef6-b8e6-83d27ef6bc4f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw"
Jul 30 00:44:44.322948 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.322844    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtzvd\" (UniqueName: \"kubernetes.io/projected/0a948f44-c233-4ef6-b8e6-83d27ef6bc4f-kube-api-access-rtzvd\") pod \"cluster-proxy-proxy-agent-f9d955c7c-j87fw\" (UID: \"0a948f44-c233-4ef6-b8e6-83d27ef6bc4f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw"
Jul 30 00:44:44.322948 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.322876    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a948f44-c233-4ef6-b8e6-83d27ef6bc4f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f9d955c7c-j87fw\" (UID: \"0a948f44-c233-4ef6-b8e6-83d27ef6bc4f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw"
Jul 30 00:44:44.322948 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.322918    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0a948f44-c233-4ef6-b8e6-83d27ef6bc4f-hub\") pod \"cluster-proxy-proxy-agent-f9d955c7c-j87fw\" (UID: \"0a948f44-c233-4ef6-b8e6-83d27ef6bc4f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw"
Jul 30 00:44:44.323152 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.322953    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0a948f44-c233-4ef6-b8e6-83d27ef6bc4f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f9d955c7c-j87fw\" (UID: \"0a948f44-c233-4ef6-b8e6-83d27ef6bc4f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw"
Jul 30 00:44:44.323656 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.323627    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a948f44-c233-4ef6-b8e6-83d27ef6bc4f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f9d955c7c-j87fw\" (UID: \"0a948f44-c233-4ef6-b8e6-83d27ef6bc4f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw"
Jul 30 00:44:44.325195 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.325169    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0a948f44-c233-4ef6-b8e6-83d27ef6bc4f-ca\") pod \"cluster-proxy-proxy-agent-f9d955c7c-j87fw\" (UID: \"0a948f44-c233-4ef6-b8e6-83d27ef6bc4f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw"
Jul 30 00:44:44.325399 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.325375    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0a948f44-c233-4ef6-b8e6-83d27ef6bc4f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f9d955c7c-j87fw\" (UID: \"0a948f44-c233-4ef6-b8e6-83d27ef6bc4f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw"
Jul 30 00:44:44.325463 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.325396    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0a948f44-c233-4ef6-b8e6-83d27ef6bc4f-hub\") pod \"cluster-proxy-proxy-agent-f9d955c7c-j87fw\" (UID: \"0a948f44-c233-4ef6-b8e6-83d27ef6bc4f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw"
Jul 30 00:44:44.325463 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.325414    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0a948f44-c233-4ef6-b8e6-83d27ef6bc4f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f9d955c7c-j87fw\" (UID: \"0a948f44-c233-4ef6-b8e6-83d27ef6bc4f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw"
Jul 30 00:44:44.348491 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.348460    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtzvd\" (UniqueName: \"kubernetes.io/projected/0a948f44-c233-4ef6-b8e6-83d27ef6bc4f-kube-api-access-rtzvd\") pod \"cluster-proxy-proxy-agent-f9d955c7c-j87fw\" (UID: \"0a948f44-c233-4ef6-b8e6-83d27ef6bc4f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw"
Jul 30 00:44:44.442540 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.442443    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw"
Jul 30 00:44:44.573500 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.573467    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw"]
Jul 30 00:44:44.576673 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:44:44.576646    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a948f44_c233_4ef6_b8e6_83d27ef6bc4f.slice/crio-fb716f264c5fb3c0a5ab33776194496f512cab5642343906ea8c4953799a0d33 WatchSource:0}: Error finding container fb716f264c5fb3c0a5ab33776194496f512cab5642343906ea8c4953799a0d33: Status 404 returned error can't find the container with id fb716f264c5fb3c0a5ab33776194496f512cab5642343906ea8c4953799a0d33
Jul 30 00:44:44.781738 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.781701    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw" event={"ID":"0a948f44-c233-4ef6-b8e6-83d27ef6bc4f","Type":"ContainerStarted","Data":"fb716f264c5fb3c0a5ab33776194496f512cab5642343906ea8c4953799a0d33"}
Jul 30 00:44:44.783593 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.783564    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4xhnp" event={"ID":"4d56a868-73af-4d44-946e-13aa14d16532","Type":"ContainerStarted","Data":"a557a1a1a218178509554480e7afce8d3e4be39cb0021635df9471a2069b7d31"}
Jul 30 00:44:44.783754 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.783625    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4xhnp" event={"ID":"4d56a868-73af-4d44-946e-13aa14d16532","Type":"ContainerStarted","Data":"a81e329d0e9633f851cf0636f988651bba653592d29ac20ad8e58ea58888c4c3"}
Jul 30 00:44:44.783869 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.783847    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4xhnp"
Jul 30 00:44:44.801161 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:44.801108    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4xhnp" podStartSLOduration=33.625029604 podStartE2EDuration="36.801092271s" totalImagesPullingTime="3.176062667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:44.799728708 +0000 UTC m=+72.022462526" watchObservedRunningTime="2026-07-30 00:44:44.801092271 +0000 UTC m=+72.023826123"
Jul 30 00:44:45.130798 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:45.130684    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9254a1f8-d810-4bea-837c-2698526cb5d4-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-58x9b\" (UID: \"9254a1f8-d810-4bea-837c-2698526cb5d4\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-58x9b"
Jul 30 00:44:45.133475 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:45.133445    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9254a1f8-d810-4bea-837c-2698526cb5d4-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-58x9b\" (UID: \"9254a1f8-d810-4bea-837c-2698526cb5d4\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-58x9b"
Jul 30 00:44:45.335589 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:45.335554    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2pjhh\""
Jul 30 00:44:45.343773 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:45.343744    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-58x9b"
Jul 30 00:44:45.500982 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:45.500282    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-58x9b"]
Jul 30 00:44:45.504894 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:44:45.504854    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9254a1f8_d810_4bea_837c_2698526cb5d4.slice/crio-6d18810972867f8edac56a99f34d051dbb275efadbbec086348317dc87e66132 WatchSource:0}: Error finding container 6d18810972867f8edac56a99f34d051dbb275efadbbec086348317dc87e66132: Status 404 returned error can't find the container with id 6d18810972867f8edac56a99f34d051dbb275efadbbec086348317dc87e66132
Jul 30 00:44:45.794946 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:45.794903    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-58x9b" event={"ID":"9254a1f8-d810-4bea-837c-2698526cb5d4","Type":"ContainerStarted","Data":"6d18810972867f8edac56a99f34d051dbb275efadbbec086348317dc87e66132"}
Jul 30 00:44:48.808234 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:48.808135    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw" event={"ID":"0a948f44-c233-4ef6-b8e6-83d27ef6bc4f","Type":"ContainerStarted","Data":"879c8e8ab859db9503a75b62a80d0f90863f2d974e33f21b24c2f77dcda1f6cb"}
Jul 30 00:44:48.808234 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:48.808187    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw" event={"ID":"0a948f44-c233-4ef6-b8e6-83d27ef6bc4f","Type":"ContainerStarted","Data":"f1970a226c8af05d42cd100165d2fbc3db8b45459c3301d4e2903689efab4835"}
Jul 30 00:44:48.808234 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:48.808205    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw" event={"ID":"0a948f44-c233-4ef6-b8e6-83d27ef6bc4f","Type":"ContainerStarted","Data":"39c6f150e49d6aa37e7fa5d712f7a4577b704ec242c37ff248e52a8f2eb44146"}
Jul 30 00:44:48.809571 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:48.809547    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-58x9b" event={"ID":"9254a1f8-d810-4bea-837c-2698526cb5d4","Type":"ContainerStarted","Data":"7ededb43b6b26ac6db7c47716683d59c427debeeb1b48148b6b441b1f091693a"}
Jul 30 00:44:48.826340 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:48.826296    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f9d955c7c-j87fw" podStartSLOduration=1.138762759 podStartE2EDuration="4.826282861s" totalImagesPullingTime="3.687520102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:48.825203453 +0000 UTC m=+76.047937300" watchObservedRunningTime="2026-07-30 00:44:48.826282861 +0000 UTC m=+76.049016681"
Jul 30 00:44:48.839167 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:48.839128    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-58x9b" podStartSLOduration=33.090428961 podStartE2EDuration="35.839114615s" totalImagesPullingTime="2.748685654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:44:48.838402473 +0000 UTC m=+76.061136294" watchObservedRunningTime="2026-07-30 00:44:48.839114615 +0000 UTC m=+76.061848434"
Jul 30 00:44:53.300303 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:53.300269    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42841174-db24-4585-a998-29845763f1e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:53.302526 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:53.302496    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42841174-db24-4585-a998-29845763f1e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jhdjl\" (UID: \"42841174-db24-4585-a998-29845763f1e4\") " pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:53.459400 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:53.459366    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-p9rd8\""
Jul 30 00:44:53.467393 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:53.467371    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jhdjl"
Jul 30 00:44:53.592801 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:53.592726    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jhdjl"]
Jul 30 00:44:53.595926 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:44:53.595897    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42841174_db24_4585_a998_29845763f1e4.slice/crio-f1fb8a7f55330768e5e5c0c0744a0c7dd67beffc52e7edd012c9b3ea079835f3 WatchSource:0}: Error finding container f1fb8a7f55330768e5e5c0c0744a0c7dd67beffc52e7edd012c9b3ea079835f3: Status 404 returned error can't find the container with id f1fb8a7f55330768e5e5c0c0744a0c7dd67beffc52e7edd012c9b3ea079835f3
Jul 30 00:44:53.826516 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:53.826479    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jhdjl" event={"ID":"42841174-db24-4585-a998-29845763f1e4","Type":"ContainerStarted","Data":"f1fb8a7f55330768e5e5c0c0744a0c7dd67beffc52e7edd012c9b3ea079835f3"}
Jul 30 00:44:54.797378 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:54.797259    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4xhnp"
Jul 30 00:44:57.352114 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:57.352078    2546 scope.go:117] "RemoveContainer" containerID="d4180a0ba92c18b1d0f8cdd3a83d32fa68b2a44dd12bf6c05551cee7642c3c07"
Jul 30 00:44:57.352624 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:57.352275    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-wx49h_openshift-console-operator(c56ae285-bf61-4f11-b846-827ea2e7594e)\"" pod="openshift-console-operator/console-operator-584dcb6b54-wx49h" podUID="c56ae285-bf61-4f11-b846-827ea2e7594e"
Jul 30 00:44:58.179138 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.179106    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ktk26"]
Jul 30 00:44:58.184456 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.184431    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.186937 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.186896    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sjmm2\""
Jul 30 00:44:58.187048 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.186946    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 30 00:44:58.187145 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.187128    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 30 00:44:58.190287 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.188685    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 30 00:44:58.190287 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.188727    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 30 00:44:58.190287 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.188959    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 30 00:44:58.190287 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.189252    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 30 00:44:58.343193 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.343153    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-metrics-client-ca\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.343384 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.343208    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-sys\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.343384 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.343273    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-node-exporter-wtmp\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.343384 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.343327    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-node-exporter-tls\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.343384 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.343358    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-root\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.343579 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.343386    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.343579 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.343432    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-469s4\" (UniqueName: \"kubernetes.io/projected/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-kube-api-access-469s4\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.343579 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.343464    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-node-exporter-accelerators-collector-config\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.343579 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.343497    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-node-exporter-textfile\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.444952 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.444864    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-node-exporter-tls\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.444952 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.444912    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-root\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.444952 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.444945    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.445502 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.444974    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-469s4\" (UniqueName: \"kubernetes.io/projected/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-kube-api-access-469s4\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.445502 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.445008    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-node-exporter-accelerators-collector-config\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.445502 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:58.445052    2546 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 30 00:44:58.445502 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.445060    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-root\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.445502 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:44:58.445138    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-node-exporter-tls podName:da33b631-3b5b-42e9-8599-7df6bcf8fc9e nodeName:}" failed. No retries permitted until 2026-07-30 00:44:58.945115399 +0000 UTC m=+86.167849195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-node-exporter-tls") pod "node-exporter-ktk26" (UID: "da33b631-3b5b-42e9-8599-7df6bcf8fc9e") : secret "node-exporter-tls" not found
Jul 30 00:44:58.445502 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.445056    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-node-exporter-textfile\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.445502 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.445263    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-metrics-client-ca\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.445502 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.445292    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-sys\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.445502 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.445314    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-node-exporter-wtmp\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.445502 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.445420    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-node-exporter-textfile\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.445502 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.445420    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-sys\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.445502 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.445475    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-node-exporter-wtmp\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.446006 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.445683    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-node-exporter-accelerators-collector-config\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.446006 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.445833    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-metrics-client-ca\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.447553 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.447514    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.456155 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.456127    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-469s4\" (UniqueName: \"kubernetes.io/projected/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-kube-api-access-469s4\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.846007 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.845969    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jhdjl" event={"ID":"42841174-db24-4585-a998-29845763f1e4","Type":"ContainerStarted","Data":"712f5b9ae538333eebdb473d6f21b375f4dad54abc20db4cdf508e411ddc6cd9"}
Jul 30 00:44:58.950652 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.950605    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-node-exporter-tls\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:58.952944 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:58.952910    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/da33b631-3b5b-42e9-8599-7df6bcf8fc9e-node-exporter-tls\") pod \"node-exporter-ktk26\" (UID: \"da33b631-3b5b-42e9-8599-7df6bcf8fc9e\") " pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:59.095472 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:59.095434    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ktk26"
Jul 30 00:44:59.106574 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:44:59.106534    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda33b631_3b5b_42e9_8599_7df6bcf8fc9e.slice/crio-85d9686b7216aa7888dbac8662d0cedfb34a6e03febfde001376fb2ea7a68f5c WatchSource:0}: Error finding container 85d9686b7216aa7888dbac8662d0cedfb34a6e03febfde001376fb2ea7a68f5c: Status 404 returned error can't find the container with id 85d9686b7216aa7888dbac8662d0cedfb34a6e03febfde001376fb2ea7a68f5c
Jul 30 00:44:59.852338 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:44:59.852264    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ktk26" event={"ID":"da33b631-3b5b-42e9-8599-7df6bcf8fc9e","Type":"ContainerStarted","Data":"85d9686b7216aa7888dbac8662d0cedfb34a6e03febfde001376fb2ea7a68f5c"}
Jul 30 00:45:00.857618 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:00.857578    2546 generic.go:356] "Generic (PLEG): container finished" podID="da33b631-3b5b-42e9-8599-7df6bcf8fc9e" containerID="09ac83ccb4b8b038e223bd5bd096abc9dc2385a39c3fa8facc7316894c22fca9" exitCode=0
Jul 30 00:45:00.858096 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:00.857637    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ktk26" event={"ID":"da33b631-3b5b-42e9-8599-7df6bcf8fc9e","Type":"ContainerDied","Data":"09ac83ccb4b8b038e223bd5bd096abc9dc2385a39c3fa8facc7316894c22fca9"}
Jul 30 00:45:02.870405 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:02.870369    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ktk26" event={"ID":"da33b631-3b5b-42e9-8599-7df6bcf8fc9e","Type":"ContainerStarted","Data":"ee59b3b262c2c6baea2370f96d8bfe5a48f479c3b2404879be35141a8aa39d8b"}
Jul 30 00:45:02.870923 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:02.870457    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ktk26" event={"ID":"da33b631-3b5b-42e9-8599-7df6bcf8fc9e","Type":"ContainerStarted","Data":"9356f351a58b2bd837f436be320c7d89ce598ef4162fc2561638e5d73eb867af"}
Jul 30 00:45:02.872139 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:02.872108    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jhdjl" event={"ID":"42841174-db24-4585-a998-29845763f1e4","Type":"ContainerStarted","Data":"1b47c405313e266465659d9ff3e01c964e2611ad6e673c902cc6ab9a0dc819be"}
Jul 30 00:45:02.893100 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:02.893052    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ktk26" podStartSLOduration=3.894934855 podStartE2EDuration="4.893038826s" totalImagesPullingTime="936.319096ms" totalInitContainerRuntime="61.784875ms" isStatefulPod=true podCreationTimestamp="2026-07-30 00:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:45:02.891841574 +0000 UTC m=+90.114575414" watchObservedRunningTime="2026-07-30 00:45:02.893038826 +0000 UTC m=+90.115772677"
Jul 30 00:45:06.888920 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:06.888883    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jhdjl" event={"ID":"42841174-db24-4585-a998-29845763f1e4","Type":"ContainerStarted","Data":"be9b4fd304ce2bffd9425a59ca12fb6d7d7862e39cac1935d65a3e997705e13d"}
Jul 30 00:45:06.906859 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:06.906814    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jhdjl" podStartSLOduration=33.421791959 podStartE2EDuration="45.906796716s" totalImagesPullingTime="12.485004757s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 00:44:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:45:06.906005863 +0000 UTC m=+94.128739684" watchObservedRunningTime="2026-07-30 00:45:06.906796716 +0000 UTC m=+94.129530550"
Jul 30 00:45:08.352580 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:08.352549    2546 scope.go:117] "RemoveContainer" containerID="d4180a0ba92c18b1d0f8cdd3a83d32fa68b2a44dd12bf6c05551cee7642c3c07"
Jul 30 00:45:08.900998 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:08.900973    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 00:45:08.901176 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:08.901069    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-wx49h" event={"ID":"c56ae285-bf61-4f11-b846-827ea2e7594e","Type":"ContainerStarted","Data":"55984adfe6362beb9abb740df292ed4489538f0b3156e3e261f09730ce1624f7"}
Jul 30 00:45:08.901340 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:08.901308    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-wx49h"
Jul 30 00:45:08.917328 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:08.917284    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-wx49h" podStartSLOduration=50.690160582 podStartE2EDuration="55.917269876s" totalImagesPullingTime="5.227109294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:45:08.916284838 +0000 UTC m=+96.139018658" watchObservedRunningTime="2026-07-30 00:45:08.917269876 +0000 UTC m=+96.140003693"
Jul 30 00:45:08.950670 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:08.950643    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-wx49h"
Jul 30 00:45:10.762832 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:10.762796    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zhf7n"
Jul 30 00:45:26.229355 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.229319    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b7d649f6f-nk8qp"]
Jul 30 00:45:26.231783 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.231759    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:26.234375 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.234347    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 30 00:45:26.234497 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.234483    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 30 00:45:26.234588 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.234569    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 30 00:45:26.235705 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.235686    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 30 00:45:26.235821 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.235772    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wjdhw\""
Jul 30 00:45:26.235979 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.235961    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 30 00:45:26.236098 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.236081    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 30 00:45:26.236160 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.236135    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 30 00:45:26.240993 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.240970    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 30 00:45:26.241512 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.241495    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b7d649f6f-nk8qp"]
Jul 30 00:45:26.249444 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.249422    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87354e71-ae91-4c65-a6b0-c613df40438b-oauth-serving-cert\") pod \"console-7b7d649f6f-nk8qp\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") " pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:26.249642 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.249618    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87354e71-ae91-4c65-a6b0-c613df40438b-trusted-ca-bundle\") pod \"console-7b7d649f6f-nk8qp\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") " pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:26.249753 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.249653    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw7s9\" (UniqueName: \"kubernetes.io/projected/87354e71-ae91-4c65-a6b0-c613df40438b-kube-api-access-rw7s9\") pod \"console-7b7d649f6f-nk8qp\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") " pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:26.249753 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.249721    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87354e71-ae91-4c65-a6b0-c613df40438b-console-serving-cert\") pod \"console-7b7d649f6f-nk8qp\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") " pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:26.249833 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.249764    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87354e71-ae91-4c65-a6b0-c613df40438b-console-config\") pod \"console-7b7d649f6f-nk8qp\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") " pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:26.249833 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.249790    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87354e71-ae91-4c65-a6b0-c613df40438b-console-oauth-config\") pod \"console-7b7d649f6f-nk8qp\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") " pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:26.249906 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.249828    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87354e71-ae91-4c65-a6b0-c613df40438b-service-ca\") pod \"console-7b7d649f6f-nk8qp\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") " pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:26.350601 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.350564    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87354e71-ae91-4c65-a6b0-c613df40438b-console-serving-cert\") pod \"console-7b7d649f6f-nk8qp\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") " pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:26.350601 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.350600    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87354e71-ae91-4c65-a6b0-c613df40438b-console-config\") pod \"console-7b7d649f6f-nk8qp\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") " pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:26.350862 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.350625    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87354e71-ae91-4c65-a6b0-c613df40438b-console-oauth-config\") pod \"console-7b7d649f6f-nk8qp\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") " pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:26.350862 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.350720    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87354e71-ae91-4c65-a6b0-c613df40438b-service-ca\") pod \"console-7b7d649f6f-nk8qp\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") " pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:26.350862 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.350799    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87354e71-ae91-4c65-a6b0-c613df40438b-oauth-serving-cert\") pod \"console-7b7d649f6f-nk8qp\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") " pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:26.350862 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.350851    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87354e71-ae91-4c65-a6b0-c613df40438b-trusted-ca-bundle\") pod \"console-7b7d649f6f-nk8qp\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") " pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:26.351060 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.350956    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw7s9\" (UniqueName: \"kubernetes.io/projected/87354e71-ae91-4c65-a6b0-c613df40438b-kube-api-access-rw7s9\") pod \"console-7b7d649f6f-nk8qp\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") " pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:26.351565 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.351544    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87354e71-ae91-4c65-a6b0-c613df40438b-oauth-serving-cert\") pod \"console-7b7d649f6f-nk8qp\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") " pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:26.351687 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.351575    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87354e71-ae91-4c65-a6b0-c613df40438b-service-ca\") pod \"console-7b7d649f6f-nk8qp\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") " pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:26.351687 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.351594    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87354e71-ae91-4c65-a6b0-c613df40438b-trusted-ca-bundle\") pod \"console-7b7d649f6f-nk8qp\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") " pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:26.352030 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.352005    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87354e71-ae91-4c65-a6b0-c613df40438b-console-config\") pod \"console-7b7d649f6f-nk8qp\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") " pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:26.353158 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.353128    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87354e71-ae91-4c65-a6b0-c613df40438b-console-serving-cert\") pod \"console-7b7d649f6f-nk8qp\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") " pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:26.353237 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.353201    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87354e71-ae91-4c65-a6b0-c613df40438b-console-oauth-config\") pod \"console-7b7d649f6f-nk8qp\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") " pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:26.359315 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.359292    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw7s9\" (UniqueName: \"kubernetes.io/projected/87354e71-ae91-4c65-a6b0-c613df40438b-kube-api-access-rw7s9\") pod \"console-7b7d649f6f-nk8qp\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") " pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:26.542643 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.542546    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:26.667440 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.667408    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b7d649f6f-nk8qp"]
Jul 30 00:45:26.670185 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:45:26.670154    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87354e71_ae91_4c65_a6b0_c613df40438b.slice/crio-548bcd88b2817c13d7f1b64bd3d188aa62214405799a9a0e09ba33dfd96a09ca WatchSource:0}: Error finding container 548bcd88b2817c13d7f1b64bd3d188aa62214405799a9a0e09ba33dfd96a09ca: Status 404 returned error can't find the container with id 548bcd88b2817c13d7f1b64bd3d188aa62214405799a9a0e09ba33dfd96a09ca
Jul 30 00:45:26.966862 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:26.966831    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b7d649f6f-nk8qp" event={"ID":"87354e71-ae91-4c65-a6b0-c613df40438b","Type":"ContainerStarted","Data":"548bcd88b2817c13d7f1b64bd3d188aa62214405799a9a0e09ba33dfd96a09ca"}
Jul 30 00:45:29.978842 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:29.978802    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b7d649f6f-nk8qp" event={"ID":"87354e71-ae91-4c65-a6b0-c613df40438b","Type":"ContainerStarted","Data":"97cdb2e7963f9ed4112800fcf22d176c4104799b90f788f89e98e8ee1a3246e2"}
Jul 30 00:45:29.995240 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:29.995192    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b7d649f6f-nk8qp" podStartSLOduration=1.04499471 podStartE2EDuration="3.995177594s" totalImagesPullingTime="2.950182884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:45:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:45:29.994517493 +0000 UTC m=+117.217251312" watchObservedRunningTime="2026-07-30 00:45:29.995177594 +0000 UTC m=+117.217911411"
Jul 30 00:45:36.543165 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:36.543121    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:36.543734 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:36.543182    2546 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:36.548225 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:36.548200    2546 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:37.010631 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:37.010597    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:45:46.039830 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:46.039794    2546 generic.go:356] "Generic (PLEG): container finished" podID="8fd5e744-a7db-428d-9d46-c2413213a511" containerID="73335d160264ac29ab155b73d69c5c60a3dd6ab27c33d8ffe93c8a9a33b8e95b" exitCode=0
Jul 30 00:45:46.040306 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:46.039869    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-8qbqm" event={"ID":"8fd5e744-a7db-428d-9d46-c2413213a511","Type":"ContainerDied","Data":"73335d160264ac29ab155b73d69c5c60a3dd6ab27c33d8ffe93c8a9a33b8e95b"}
Jul 30 00:45:46.040306 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:46.040276    2546 scope.go:117] "RemoveContainer" containerID="73335d160264ac29ab155b73d69c5c60a3dd6ab27c33d8ffe93c8a9a33b8e95b"
Jul 30 00:45:46.041267 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:46.041246    2546 generic.go:356] "Generic (PLEG): container finished" podID="3fec0f2e-4a5a-4a55-89ce-4273ee8a5669" containerID="5320258ad0c7dd191411548e99785c3b8ac8a99b8eeda1be0117a0f4f90fee4a" exitCode=0
Jul 30 00:45:46.041371 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:46.041297    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-4qsk7" event={"ID":"3fec0f2e-4a5a-4a55-89ce-4273ee8a5669","Type":"ContainerDied","Data":"5320258ad0c7dd191411548e99785c3b8ac8a99b8eeda1be0117a0f4f90fee4a"}
Jul 30 00:45:46.041666 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:46.041651    2546 scope.go:117] "RemoveContainer" containerID="5320258ad0c7dd191411548e99785c3b8ac8a99b8eeda1be0117a0f4f90fee4a"
Jul 30 00:45:47.046973 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:47.046938    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-8qbqm" event={"ID":"8fd5e744-a7db-428d-9d46-c2413213a511","Type":"ContainerStarted","Data":"cdf59c95a37e042340965870ad484c4c257488c565871cc61faecbb1a31ec235"}
Jul 30 00:45:47.048648 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:45:47.048619    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-4qsk7" event={"ID":"3fec0f2e-4a5a-4a55-89ce-4273ee8a5669","Type":"ContainerStarted","Data":"f392a19b3032a3c24db49bbdb8dec7b88b1a346b37c5b12d998e01e1e884e7e8"}
Jul 30 00:46:38.589212 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:46:38.589130    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b7d649f6f-nk8qp"]
Jul 30 00:47:03.609876 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:03.609802    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7b7d649f6f-nk8qp" podUID="87354e71-ae91-4c65-a6b0-c613df40438b" containerName="console" containerID="cri-o://97cdb2e7963f9ed4112800fcf22d176c4104799b90f788f89e98e8ee1a3246e2" gracePeriod=15
Jul 30 00:47:03.862851 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:03.862785    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:47:03.875881 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:03.875854    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87354e71-ae91-4c65-a6b0-c613df40438b-console-oauth-config\") pod \"87354e71-ae91-4c65-a6b0-c613df40438b\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") "
Jul 30 00:47:03.876093 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:03.875894    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rw7s9\" (UniqueName: \"kubernetes.io/projected/87354e71-ae91-4c65-a6b0-c613df40438b-kube-api-access-rw7s9\") pod \"87354e71-ae91-4c65-a6b0-c613df40438b\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") "
Jul 30 00:47:03.876093 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:03.875924    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87354e71-ae91-4c65-a6b0-c613df40438b-trusted-ca-bundle\") pod \"87354e71-ae91-4c65-a6b0-c613df40438b\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") "
Jul 30 00:47:03.876093 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:03.875958    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87354e71-ae91-4c65-a6b0-c613df40438b-console-config\") pod \"87354e71-ae91-4c65-a6b0-c613df40438b\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") "
Jul 30 00:47:03.876093 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:03.875973    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87354e71-ae91-4c65-a6b0-c613df40438b-service-ca\") pod \"87354e71-ae91-4c65-a6b0-c613df40438b\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") "
Jul 30 00:47:03.876093 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:03.876019    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87354e71-ae91-4c65-a6b0-c613df40438b-oauth-serving-cert\") pod \"87354e71-ae91-4c65-a6b0-c613df40438b\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") "
Jul 30 00:47:03.876093 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:03.876040    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87354e71-ae91-4c65-a6b0-c613df40438b-console-serving-cert\") pod \"87354e71-ae91-4c65-a6b0-c613df40438b\" (UID: \"87354e71-ae91-4c65-a6b0-c613df40438b\") "
Jul 30 00:47:03.876584 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:03.876554    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87354e71-ae91-4c65-a6b0-c613df40438b-service-ca" (OuterVolumeSpecName: "service-ca") pod "87354e71-ae91-4c65-a6b0-c613df40438b" (UID: "87354e71-ae91-4c65-a6b0-c613df40438b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 00:47:03.876584 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:03.876571    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87354e71-ae91-4c65-a6b0-c613df40438b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "87354e71-ae91-4c65-a6b0-c613df40438b" (UID: "87354e71-ae91-4c65-a6b0-c613df40438b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 00:47:03.876716 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:03.876566    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87354e71-ae91-4c65-a6b0-c613df40438b-console-config" (OuterVolumeSpecName: "console-config") pod "87354e71-ae91-4c65-a6b0-c613df40438b" (UID: "87354e71-ae91-4c65-a6b0-c613df40438b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 00:47:03.876773 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:03.876744    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87354e71-ae91-4c65-a6b0-c613df40438b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "87354e71-ae91-4c65-a6b0-c613df40438b" (UID: "87354e71-ae91-4c65-a6b0-c613df40438b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 00:47:03.878324 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:03.878302    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87354e71-ae91-4c65-a6b0-c613df40438b-kube-api-access-rw7s9" (OuterVolumeSpecName: "kube-api-access-rw7s9") pod "87354e71-ae91-4c65-a6b0-c613df40438b" (UID: "87354e71-ae91-4c65-a6b0-c613df40438b"). InnerVolumeSpecName "kube-api-access-rw7s9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 00:47:03.878431 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:03.878318    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87354e71-ae91-4c65-a6b0-c613df40438b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "87354e71-ae91-4c65-a6b0-c613df40438b" (UID: "87354e71-ae91-4c65-a6b0-c613df40438b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 00:47:03.878431 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:03.878355    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87354e71-ae91-4c65-a6b0-c613df40438b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "87354e71-ae91-4c65-a6b0-c613df40438b" (UID: "87354e71-ae91-4c65-a6b0-c613df40438b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 00:47:03.976822 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:03.976783    2546 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87354e71-ae91-4c65-a6b0-c613df40438b-console-oauth-config\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:47:03.976822 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:03.976812    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rw7s9\" (UniqueName: \"kubernetes.io/projected/87354e71-ae91-4c65-a6b0-c613df40438b-kube-api-access-rw7s9\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:47:03.976822 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:03.976824    2546 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87354e71-ae91-4c65-a6b0-c613df40438b-trusted-ca-bundle\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:47:03.976822 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:03.976834    2546 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87354e71-ae91-4c65-a6b0-c613df40438b-console-config\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:47:03.977107 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:03.976843    2546 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87354e71-ae91-4c65-a6b0-c613df40438b-service-ca\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:47:03.977107 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:03.976852    2546 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87354e71-ae91-4c65-a6b0-c613df40438b-oauth-serving-cert\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:47:03.977107 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:03.976863    2546 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87354e71-ae91-4c65-a6b0-c613df40438b-console-serving-cert\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:47:04.325925 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:04.325888    2546 generic.go:356] "Generic (PLEG): container finished" podID="87354e71-ae91-4c65-a6b0-c613df40438b" containerID="97cdb2e7963f9ed4112800fcf22d176c4104799b90f788f89e98e8ee1a3246e2" exitCode=0
Jul 30 00:47:04.326124 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:04.325937    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b7d649f6f-nk8qp"
Jul 30 00:47:04.326124 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:04.325949    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b7d649f6f-nk8qp" event={"ID":"87354e71-ae91-4c65-a6b0-c613df40438b","Type":"ContainerDied","Data":"97cdb2e7963f9ed4112800fcf22d176c4104799b90f788f89e98e8ee1a3246e2"}
Jul 30 00:47:04.326124 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:04.325989    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b7d649f6f-nk8qp" event={"ID":"87354e71-ae91-4c65-a6b0-c613df40438b","Type":"ContainerDied","Data":"548bcd88b2817c13d7f1b64bd3d188aa62214405799a9a0e09ba33dfd96a09ca"}
Jul 30 00:47:04.326124 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:04.326011    2546 scope.go:117] "RemoveContainer" containerID="97cdb2e7963f9ed4112800fcf22d176c4104799b90f788f89e98e8ee1a3246e2"
Jul 30 00:47:04.334790 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:04.334766    2546 scope.go:117] "RemoveContainer" containerID="97cdb2e7963f9ed4112800fcf22d176c4104799b90f788f89e98e8ee1a3246e2"
Jul 30 00:47:04.335110 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:47:04.335082    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97cdb2e7963f9ed4112800fcf22d176c4104799b90f788f89e98e8ee1a3246e2\": container with ID starting with 97cdb2e7963f9ed4112800fcf22d176c4104799b90f788f89e98e8ee1a3246e2 not found: ID does not exist" containerID="97cdb2e7963f9ed4112800fcf22d176c4104799b90f788f89e98e8ee1a3246e2"
Jul 30 00:47:04.335166 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:04.335122    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97cdb2e7963f9ed4112800fcf22d176c4104799b90f788f89e98e8ee1a3246e2"} err="failed to get container status \"97cdb2e7963f9ed4112800fcf22d176c4104799b90f788f89e98e8ee1a3246e2\": rpc error: code = NotFound desc = could not find container \"97cdb2e7963f9ed4112800fcf22d176c4104799b90f788f89e98e8ee1a3246e2\": container with ID starting with 97cdb2e7963f9ed4112800fcf22d176c4104799b90f788f89e98e8ee1a3246e2 not found: ID does not exist"
Jul 30 00:47:04.346870 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:04.346837    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b7d649f6f-nk8qp"]
Jul 30 00:47:04.350846 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:04.350820    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b7d649f6f-nk8qp"]
Jul 30 00:47:05.357415 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:47:05.357377    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87354e71-ae91-4c65-a6b0-c613df40438b" path="/var/lib/kubelet/pods/87354e71-ae91-4c65-a6b0-c613df40438b/volumes"
Jul 30 00:48:33.151089 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:48:33.151057    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 00:48:33.154569 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:48:33.154546    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 00:48:33.162162 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:48:33.162135    2546 kubelet.go:1648] "Image garbage collection succeeded"
Jul 30 00:49:22.372064 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:22.372026    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-mfmmf"]
Jul 30 00:49:22.372545 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:22.372281    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87354e71-ae91-4c65-a6b0-c613df40438b" containerName="console"
Jul 30 00:49:22.372545 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:22.372292    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="87354e71-ae91-4c65-a6b0-c613df40438b" containerName="console"
Jul 30 00:49:22.372545 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:22.372346    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="87354e71-ae91-4c65-a6b0-c613df40438b" containerName="console"
Jul 30 00:49:22.375080 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:22.375062    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mfmmf"
Jul 30 00:49:22.378156 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:22.378130    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 30 00:49:22.378276 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:22.378193    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 30 00:49:22.378984 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:22.378960    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-6mwfs\""
Jul 30 00:49:22.392017 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:22.389295    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-mfmmf"]
Jul 30 00:49:22.461871 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:22.461832    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns8ts\" (UniqueName: \"kubernetes.io/projected/ae1ecd13-569a-43ed-b529-ac6bcc916e1e-kube-api-access-ns8ts\") pod \"cert-manager-cainjector-559cdd8c87-mfmmf\" (UID: \"ae1ecd13-569a-43ed-b529-ac6bcc916e1e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mfmmf"
Jul 30 00:49:22.462040 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:22.461885    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae1ecd13-569a-43ed-b529-ac6bcc916e1e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-mfmmf\" (UID: \"ae1ecd13-569a-43ed-b529-ac6bcc916e1e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mfmmf"
Jul 30 00:49:22.563113 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:22.563076    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns8ts\" (UniqueName: \"kubernetes.io/projected/ae1ecd13-569a-43ed-b529-ac6bcc916e1e-kube-api-access-ns8ts\") pod \"cert-manager-cainjector-559cdd8c87-mfmmf\" (UID: \"ae1ecd13-569a-43ed-b529-ac6bcc916e1e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mfmmf"
Jul 30 00:49:22.563261 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:22.563125    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae1ecd13-569a-43ed-b529-ac6bcc916e1e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-mfmmf\" (UID: \"ae1ecd13-569a-43ed-b529-ac6bcc916e1e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mfmmf"
Jul 30 00:49:22.571961 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:22.571940    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae1ecd13-569a-43ed-b529-ac6bcc916e1e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-mfmmf\" (UID: \"ae1ecd13-569a-43ed-b529-ac6bcc916e1e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mfmmf"
Jul 30 00:49:22.572090 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:22.572071    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns8ts\" (UniqueName: \"kubernetes.io/projected/ae1ecd13-569a-43ed-b529-ac6bcc916e1e-kube-api-access-ns8ts\") pod \"cert-manager-cainjector-559cdd8c87-mfmmf\" (UID: \"ae1ecd13-569a-43ed-b529-ac6bcc916e1e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mfmmf"
Jul 30 00:49:22.684683 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:22.684602    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mfmmf"
Jul 30 00:49:22.807636 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:22.807582    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-mfmmf"]
Jul 30 00:49:22.810439 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:49:22.810411    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae1ecd13_569a_43ed_b529_ac6bcc916e1e.slice/crio-2e7cc63b72a33d8a77fbac2d250847752ffa93f62f588d15afe0e506f8450ad6 WatchSource:0}: Error finding container 2e7cc63b72a33d8a77fbac2d250847752ffa93f62f588d15afe0e506f8450ad6: Status 404 returned error can't find the container with id 2e7cc63b72a33d8a77fbac2d250847752ffa93f62f588d15afe0e506f8450ad6
Jul 30 00:49:22.812693 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:22.812677    2546 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 00:49:23.809420 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:23.809380    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mfmmf" event={"ID":"ae1ecd13-569a-43ed-b529-ac6bcc916e1e","Type":"ContainerStarted","Data":"2e7cc63b72a33d8a77fbac2d250847752ffa93f62f588d15afe0e506f8450ad6"}
Jul 30 00:49:26.822925 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:26.822890    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mfmmf" event={"ID":"ae1ecd13-569a-43ed-b529-ac6bcc916e1e","Type":"ContainerStarted","Data":"396c27c5f73da70a4b3b16029b98221555698333d72ff105a391ac240cb66a04"}
Jul 30 00:49:26.840189 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:26.840136    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mfmmf" podStartSLOduration=1.692826868 podStartE2EDuration="4.840119729s" totalImagesPullingTime="3.147292861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:49:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:49:26.839129707 +0000 UTC m=+354.061863524" watchObservedRunningTime="2026-07-30 00:49:26.840119729 +0000 UTC m=+354.062853545"
Jul 30 00:49:32.311944 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:32.311911    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xslf4"]
Jul 30 00:49:32.315083 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:32.315067    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xslf4"
Jul 30 00:49:32.317612 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:32.317591    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-fkpt9\""
Jul 30 00:49:32.317718 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:32.317637    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 30 00:49:32.318772 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:32.318757    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 30 00:49:32.327062 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:32.327042    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xslf4"]
Jul 30 00:49:32.435267 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:32.435238    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrsfg\" (UniqueName: \"kubernetes.io/projected/a92e23ed-7c35-45fc-8121-69918812e3bb-kube-api-access-wrsfg\") pod \"openshift-lws-operator-fd8ccff4c-xslf4\" (UID: \"a92e23ed-7c35-45fc-8121-69918812e3bb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xslf4"
Jul 30 00:49:32.435400 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:32.435300    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a92e23ed-7c35-45fc-8121-69918812e3bb-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xslf4\" (UID: \"a92e23ed-7c35-45fc-8121-69918812e3bb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xslf4"
Jul 30 00:49:32.536079 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:32.536050    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrsfg\" (UniqueName: \"kubernetes.io/projected/a92e23ed-7c35-45fc-8121-69918812e3bb-kube-api-access-wrsfg\") pod \"openshift-lws-operator-fd8ccff4c-xslf4\" (UID: \"a92e23ed-7c35-45fc-8121-69918812e3bb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xslf4"
Jul 30 00:49:32.536204 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:32.536088    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a92e23ed-7c35-45fc-8121-69918812e3bb-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xslf4\" (UID: \"a92e23ed-7c35-45fc-8121-69918812e3bb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xslf4"
Jul 30 00:49:32.536424 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:32.536408    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a92e23ed-7c35-45fc-8121-69918812e3bb-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xslf4\" (UID: \"a92e23ed-7c35-45fc-8121-69918812e3bb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xslf4"
Jul 30 00:49:32.544808 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:32.544787    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrsfg\" (UniqueName: \"kubernetes.io/projected/a92e23ed-7c35-45fc-8121-69918812e3bb-kube-api-access-wrsfg\") pod \"openshift-lws-operator-fd8ccff4c-xslf4\" (UID: \"a92e23ed-7c35-45fc-8121-69918812e3bb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xslf4"
Jul 30 00:49:32.623789 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:32.623713    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xslf4"
Jul 30 00:49:32.744928 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:32.744897    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xslf4"]
Jul 30 00:49:32.748200 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:49:32.748171    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda92e23ed_7c35_45fc_8121_69918812e3bb.slice/crio-5e05da13b2cca25994f72f15faedde92a183a4296e74276ec1343c1e9324025c WatchSource:0}: Error finding container 5e05da13b2cca25994f72f15faedde92a183a4296e74276ec1343c1e9324025c: Status 404 returned error can't find the container with id 5e05da13b2cca25994f72f15faedde92a183a4296e74276ec1343c1e9324025c
Jul 30 00:49:32.851254 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:32.851225    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xslf4" event={"ID":"a92e23ed-7c35-45fc-8121-69918812e3bb","Type":"ContainerStarted","Data":"5e05da13b2cca25994f72f15faedde92a183a4296e74276ec1343c1e9324025c"}
Jul 30 00:49:35.867010 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:35.866971    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xslf4" event={"ID":"a92e23ed-7c35-45fc-8121-69918812e3bb","Type":"ContainerStarted","Data":"905d5977e805e070b145cfd4d8ab942fa054ea72023473c22dfd5e54218d3038"}
Jul 30 00:49:35.881599 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:35.881554    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xslf4" podStartSLOduration=1.415760738 podStartE2EDuration="3.881540901s" totalImagesPullingTime="2.465780163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:49:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:49:35.880765211 +0000 UTC m=+363.103499029" watchObservedRunningTime="2026-07-30 00:49:35.881540901 +0000 UTC m=+363.104274708"
Jul 30 00:49:54.081156 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.081117    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj"]
Jul 30 00:49:54.099364 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.099338    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj"
Jul 30 00:49:54.103152 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.103123    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 30 00:49:54.103313 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.103287    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 30 00:49:54.103770 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.103749    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 30 00:49:54.103862 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.103840    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 30 00:49:54.104434 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.104408    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-585w4\""
Jul 30 00:49:54.105842 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.105818    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj"]
Jul 30 00:49:54.198884 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.198841    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bdf82d5b-7f06-4b71-862d-5a9aeac97c71-webhook-cert\") pod \"opendatahub-operator-controller-manager-746b4b6684-9phwj\" (UID: \"bdf82d5b-7f06-4b71-862d-5a9aeac97c71\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj"
Jul 30 00:49:54.199104 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.198921    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdf82d5b-7f06-4b71-862d-5a9aeac97c71-tmp\") pod \"opendatahub-operator-controller-manager-746b4b6684-9phwj\" (UID: \"bdf82d5b-7f06-4b71-862d-5a9aeac97c71\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj"
Jul 30 00:49:54.199104 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.198952    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bdf82d5b-7f06-4b71-862d-5a9aeac97c71-apiservice-cert\") pod \"opendatahub-operator-controller-manager-746b4b6684-9phwj\" (UID: \"bdf82d5b-7f06-4b71-862d-5a9aeac97c71\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj"
Jul 30 00:49:54.199104 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.198980    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf6kg\" (UniqueName: \"kubernetes.io/projected/bdf82d5b-7f06-4b71-862d-5a9aeac97c71-kube-api-access-mf6kg\") pod \"opendatahub-operator-controller-manager-746b4b6684-9phwj\" (UID: \"bdf82d5b-7f06-4b71-862d-5a9aeac97c71\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj"
Jul 30 00:49:54.199104 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.199058    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bdf82d5b-7f06-4b71-862d-5a9aeac97c71-manifests\") pod \"opendatahub-operator-controller-manager-746b4b6684-9phwj\" (UID: \"bdf82d5b-7f06-4b71-862d-5a9aeac97c71\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj"
Jul 30 00:49:54.299772 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.299703    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdf82d5b-7f06-4b71-862d-5a9aeac97c71-tmp\") pod \"opendatahub-operator-controller-manager-746b4b6684-9phwj\" (UID: \"bdf82d5b-7f06-4b71-862d-5a9aeac97c71\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj"
Jul 30 00:49:54.299772 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.299778    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bdf82d5b-7f06-4b71-862d-5a9aeac97c71-apiservice-cert\") pod \"opendatahub-operator-controller-manager-746b4b6684-9phwj\" (UID: \"bdf82d5b-7f06-4b71-862d-5a9aeac97c71\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj"
Jul 30 00:49:54.300039 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.299810    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf6kg\" (UniqueName: \"kubernetes.io/projected/bdf82d5b-7f06-4b71-862d-5a9aeac97c71-kube-api-access-mf6kg\") pod \"opendatahub-operator-controller-manager-746b4b6684-9phwj\" (UID: \"bdf82d5b-7f06-4b71-862d-5a9aeac97c71\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj"
Jul 30 00:49:54.300039 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.299832    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bdf82d5b-7f06-4b71-862d-5a9aeac97c71-manifests\") pod \"opendatahub-operator-controller-manager-746b4b6684-9phwj\" (UID: \"bdf82d5b-7f06-4b71-862d-5a9aeac97c71\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj"
Jul 30 00:49:54.300039 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.299852    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bdf82d5b-7f06-4b71-862d-5a9aeac97c71-webhook-cert\") pod \"opendatahub-operator-controller-manager-746b4b6684-9phwj\" (UID: \"bdf82d5b-7f06-4b71-862d-5a9aeac97c71\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj"
Jul 30 00:49:54.300199 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.300179    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bdf82d5b-7f06-4b71-862d-5a9aeac97c71-manifests\") pod \"opendatahub-operator-controller-manager-746b4b6684-9phwj\" (UID: \"bdf82d5b-7f06-4b71-862d-5a9aeac97c71\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj"
Jul 30 00:49:54.302065 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.302033    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bdf82d5b-7f06-4b71-862d-5a9aeac97c71-webhook-cert\") pod \"opendatahub-operator-controller-manager-746b4b6684-9phwj\" (UID: \"bdf82d5b-7f06-4b71-862d-5a9aeac97c71\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj"
Jul 30 00:49:54.302150 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.302044    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdf82d5b-7f06-4b71-862d-5a9aeac97c71-tmp\") pod \"opendatahub-operator-controller-manager-746b4b6684-9phwj\" (UID: \"bdf82d5b-7f06-4b71-862d-5a9aeac97c71\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj"
Jul 30 00:49:54.302187 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.302146    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bdf82d5b-7f06-4b71-862d-5a9aeac97c71-apiservice-cert\") pod \"opendatahub-operator-controller-manager-746b4b6684-9phwj\" (UID: \"bdf82d5b-7f06-4b71-862d-5a9aeac97c71\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj"
Jul 30 00:49:54.308158 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.308134    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf6kg\" (UniqueName: \"kubernetes.io/projected/bdf82d5b-7f06-4b71-862d-5a9aeac97c71-kube-api-access-mf6kg\") pod \"opendatahub-operator-controller-manager-746b4b6684-9phwj\" (UID: \"bdf82d5b-7f06-4b71-862d-5a9aeac97c71\") " pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj"
Jul 30 00:49:54.415352 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.415263    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj"
Jul 30 00:49:54.583586 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.583553    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj"]
Jul 30 00:49:54.587445 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:49:54.587415    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdf82d5b_7f06_4b71_862d_5a9aeac97c71.slice/crio-bdcbe55e6c6e05c5f1f709f63693b4a154217525de718fd7c2038b42e59a8405 WatchSource:0}: Error finding container bdcbe55e6c6e05c5f1f709f63693b4a154217525de718fd7c2038b42e59a8405: Status 404 returned error can't find the container with id bdcbe55e6c6e05c5f1f709f63693b4a154217525de718fd7c2038b42e59a8405
Jul 30 00:49:54.939811 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:54.939780    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj" event={"ID":"bdf82d5b-7f06-4b71-862d-5a9aeac97c71","Type":"ContainerStarted","Data":"bdcbe55e6c6e05c5f1f709f63693b4a154217525de718fd7c2038b42e59a8405"}
Jul 30 00:49:57.954946 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:57.954857    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj" event={"ID":"bdf82d5b-7f06-4b71-862d-5a9aeac97c71","Type":"ContainerStarted","Data":"f05260bbef5c0da0a76a0f3e6fb30a6415020c6f02e8ee8036ee250b57a0faae"}
Jul 30 00:49:58.961187 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:58.961093    2546 generic.go:356] "Generic (PLEG): container finished" podID="bdf82d5b-7f06-4b71-862d-5a9aeac97c71" containerID="f05260bbef5c0da0a76a0f3e6fb30a6415020c6f02e8ee8036ee250b57a0faae" exitCode=0
Jul 30 00:49:58.961187 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:58.961147    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj" event={"ID":"bdf82d5b-7f06-4b71-862d-5a9aeac97c71","Type":"ContainerDied","Data":"f05260bbef5c0da0a76a0f3e6fb30a6415020c6f02e8ee8036ee250b57a0faae"}
Jul 30 00:49:59.966642 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:59.966607    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj" event={"ID":"bdf82d5b-7f06-4b71-862d-5a9aeac97c71","Type":"ContainerStarted","Data":"dc2afc818c29cfd37ad90aa608f3d7f544b9b44b6bdd4325fef566b80a7e6587"}
Jul 30 00:49:59.967038 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:59.966750    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj"
Jul 30 00:49:59.988297 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:49:59.988245    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj" podStartSLOduration=1.600790347 podStartE2EDuration="5.988229528s" totalImagesPullingTime="3.374209346s" totalInitContainerRuntime="1.013229835s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:49:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:49:59.986752291 +0000 UTC m=+387.209486119" watchObservedRunningTime="2026-07-30 00:49:59.988229528 +0000 UTC m=+387.210963345"
Jul 30 00:50:10.975674 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:10.975635    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-746b4b6684-9phwj"
Jul 30 00:50:13.795669 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:13.795632    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-95755dfb5-nf8t8"]
Jul 30 00:50:13.803876 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:13.803844    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-95755dfb5-nf8t8"
Jul 30 00:50:13.806586 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:13.806559    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 30 00:50:13.806902 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:13.806887    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 30 00:50:13.807997 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:13.807980    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-dqwg4\""
Jul 30 00:50:13.808077 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:13.808024    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 30 00:50:13.808139 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:13.808082    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 30 00:50:13.812637 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:13.812617    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-95755dfb5-nf8t8"]
Jul 30 00:50:13.954839 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:13.954800    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6jn5\" (UniqueName: \"kubernetes.io/projected/8563107f-b595-483e-aa72-9a6be121a415-kube-api-access-g6jn5\") pod \"kube-auth-proxy-95755dfb5-nf8t8\" (UID: \"8563107f-b595-483e-aa72-9a6be121a415\") " pod="openshift-ingress/kube-auth-proxy-95755dfb5-nf8t8"
Jul 30 00:50:13.955000 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:13.954885    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8563107f-b595-483e-aa72-9a6be121a415-tls-certs\") pod \"kube-auth-proxy-95755dfb5-nf8t8\" (UID: \"8563107f-b595-483e-aa72-9a6be121a415\") " pod="openshift-ingress/kube-auth-proxy-95755dfb5-nf8t8"
Jul 30 00:50:13.955000 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:13.954904    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8563107f-b595-483e-aa72-9a6be121a415-tmp\") pod \"kube-auth-proxy-95755dfb5-nf8t8\" (UID: \"8563107f-b595-483e-aa72-9a6be121a415\") " pod="openshift-ingress/kube-auth-proxy-95755dfb5-nf8t8"
Jul 30 00:50:14.055831 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:14.055735    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8563107f-b595-483e-aa72-9a6be121a415-tls-certs\") pod \"kube-auth-proxy-95755dfb5-nf8t8\" (UID: \"8563107f-b595-483e-aa72-9a6be121a415\") " pod="openshift-ingress/kube-auth-proxy-95755dfb5-nf8t8"
Jul 30 00:50:14.055831 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:14.055777    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8563107f-b595-483e-aa72-9a6be121a415-tmp\") pod \"kube-auth-proxy-95755dfb5-nf8t8\" (UID: \"8563107f-b595-483e-aa72-9a6be121a415\") " pod="openshift-ingress/kube-auth-proxy-95755dfb5-nf8t8"
Jul 30 00:50:14.055831 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:14.055811    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6jn5\" (UniqueName: \"kubernetes.io/projected/8563107f-b595-483e-aa72-9a6be121a415-kube-api-access-g6jn5\") pod \"kube-auth-proxy-95755dfb5-nf8t8\" (UID: \"8563107f-b595-483e-aa72-9a6be121a415\") " pod="openshift-ingress/kube-auth-proxy-95755dfb5-nf8t8"
Jul 30 00:50:14.058044 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:14.058017    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8563107f-b595-483e-aa72-9a6be121a415-tmp\") pod \"kube-auth-proxy-95755dfb5-nf8t8\" (UID: \"8563107f-b595-483e-aa72-9a6be121a415\") " pod="openshift-ingress/kube-auth-proxy-95755dfb5-nf8t8"
Jul 30 00:50:14.058181 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:14.058162    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8563107f-b595-483e-aa72-9a6be121a415-tls-certs\") pod \"kube-auth-proxy-95755dfb5-nf8t8\" (UID: \"8563107f-b595-483e-aa72-9a6be121a415\") " pod="openshift-ingress/kube-auth-proxy-95755dfb5-nf8t8"
Jul 30 00:50:14.074193 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:14.074172    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6jn5\" (UniqueName: \"kubernetes.io/projected/8563107f-b595-483e-aa72-9a6be121a415-kube-api-access-g6jn5\") pod \"kube-auth-proxy-95755dfb5-nf8t8\" (UID: \"8563107f-b595-483e-aa72-9a6be121a415\") " pod="openshift-ingress/kube-auth-proxy-95755dfb5-nf8t8"
Jul 30 00:50:14.097630 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:14.097598    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jt7jk"]
Jul 30 00:50:14.102036 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:14.102015    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-jt7jk"
Jul 30 00:50:14.104680 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:14.104650    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 30 00:50:14.104797 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:14.104693    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-49l8s\""
Jul 30 00:50:14.114310 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:14.114281    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-95755dfb5-nf8t8"
Jul 30 00:50:14.115094 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:14.115066    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jt7jk"]
Jul 30 00:50:14.249515 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:14.249482    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-95755dfb5-nf8t8"]
Jul 30 00:50:14.252909 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:50:14.252883    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8563107f_b595_483e_aa72_9a6be121a415.slice/crio-aa6c553e6a2f50e2907a1db09e5227353fde34a3e07a25aa8dea9ea5de0d9fe3 WatchSource:0}: Error finding container aa6c553e6a2f50e2907a1db09e5227353fde34a3e07a25aa8dea9ea5de0d9fe3: Status 404 returned error can't find the container with id aa6c553e6a2f50e2907a1db09e5227353fde34a3e07a25aa8dea9ea5de0d9fe3
Jul 30 00:50:14.258064 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:14.258041    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9pq4\" (UniqueName: \"kubernetes.io/projected/7902a946-c204-49e0-a986-fd5b0db8011c-kube-api-access-t9pq4\") pod \"odh-model-controller-7998c67dc4-jt7jk\" (UID: \"7902a946-c204-49e0-a986-fd5b0db8011c\") " pod="opendatahub/odh-model-controller-7998c67dc4-jt7jk"
Jul 30 00:50:14.258176 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:14.258159    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7902a946-c204-49e0-a986-fd5b0db8011c-cert\") pod \"odh-model-controller-7998c67dc4-jt7jk\" (UID: \"7902a946-c204-49e0-a986-fd5b0db8011c\") " pod="opendatahub/odh-model-controller-7998c67dc4-jt7jk"
Jul 30 00:50:14.358854 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:14.358763    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7902a946-c204-49e0-a986-fd5b0db8011c-cert\") pod \"odh-model-controller-7998c67dc4-jt7jk\" (UID: \"7902a946-c204-49e0-a986-fd5b0db8011c\") " pod="opendatahub/odh-model-controller-7998c67dc4-jt7jk"
Jul 30 00:50:14.358854 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:14.358819    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9pq4\" (UniqueName: \"kubernetes.io/projected/7902a946-c204-49e0-a986-fd5b0db8011c-kube-api-access-t9pq4\") pod \"odh-model-controller-7998c67dc4-jt7jk\" (UID: \"7902a946-c204-49e0-a986-fd5b0db8011c\") " pod="opendatahub/odh-model-controller-7998c67dc4-jt7jk"
Jul 30 00:50:14.359052 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:50:14.358900    2546 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 30 00:50:14.359052 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:50:14.358961    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7902a946-c204-49e0-a986-fd5b0db8011c-cert podName:7902a946-c204-49e0-a986-fd5b0db8011c nodeName:}" failed. No retries permitted until 2026-07-30 00:50:14.858945957 +0000 UTC m=+402.081679757 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7902a946-c204-49e0-a986-fd5b0db8011c-cert") pod "odh-model-controller-7998c67dc4-jt7jk" (UID: "7902a946-c204-49e0-a986-fd5b0db8011c") : secret "odh-model-controller-webhook-cert" not found
Jul 30 00:50:14.376052 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:14.376022    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9pq4\" (UniqueName: \"kubernetes.io/projected/7902a946-c204-49e0-a986-fd5b0db8011c-kube-api-access-t9pq4\") pod \"odh-model-controller-7998c67dc4-jt7jk\" (UID: \"7902a946-c204-49e0-a986-fd5b0db8011c\") " pod="opendatahub/odh-model-controller-7998c67dc4-jt7jk"
Jul 30 00:50:14.864056 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:14.864022    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7902a946-c204-49e0-a986-fd5b0db8011c-cert\") pod \"odh-model-controller-7998c67dc4-jt7jk\" (UID: \"7902a946-c204-49e0-a986-fd5b0db8011c\") " pod="opendatahub/odh-model-controller-7998c67dc4-jt7jk"
Jul 30 00:50:14.866415 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:14.866393    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7902a946-c204-49e0-a986-fd5b0db8011c-cert\") pod \"odh-model-controller-7998c67dc4-jt7jk\" (UID: \"7902a946-c204-49e0-a986-fd5b0db8011c\") " pod="opendatahub/odh-model-controller-7998c67dc4-jt7jk"
Jul 30 00:50:15.013274 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.013229    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-jt7jk"
Jul 30 00:50:15.030978 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.030948    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-95755dfb5-nf8t8" event={"ID":"8563107f-b595-483e-aa72-9a6be121a415","Type":"ContainerStarted","Data":"aa6c553e6a2f50e2907a1db09e5227353fde34a3e07a25aa8dea9ea5de0d9fe3"}
Jul 30 00:50:15.045213 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.045182    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hbwcm"]
Jul 30 00:50:15.048500 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.048479    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-hbwcm"
Jul 30 00:50:15.057450 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.057428    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 30 00:50:15.057671 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.057652    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-phkrt\""
Jul 30 00:50:15.067647 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.067616    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hbwcm"]
Jul 30 00:50:15.166499 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.166422    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2469cd23-8223-4779-a4ac-4268d35322db-tls-certs\") pod \"model-serving-api-54d89bd5fc-hbwcm\" (UID: \"2469cd23-8223-4779-a4ac-4268d35322db\") " pod="opendatahub/model-serving-api-54d89bd5fc-hbwcm"
Jul 30 00:50:15.166499 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.166458    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg5w6\" (UniqueName: \"kubernetes.io/projected/2469cd23-8223-4779-a4ac-4268d35322db-kube-api-access-zg5w6\") pod \"model-serving-api-54d89bd5fc-hbwcm\" (UID: \"2469cd23-8223-4779-a4ac-4268d35322db\") " pod="opendatahub/model-serving-api-54d89bd5fc-hbwcm"
Jul 30 00:50:15.170571 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.170178    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jt7jk"]
Jul 30 00:50:15.172769 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:50:15.172742    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7902a946_c204_49e0_a986_fd5b0db8011c.slice/crio-b2539cf40fcab80816b4fead0989f512520d3f778410acffdabfa12cfe25dc7a WatchSource:0}: Error finding container b2539cf40fcab80816b4fead0989f512520d3f778410acffdabfa12cfe25dc7a: Status 404 returned error can't find the container with id b2539cf40fcab80816b4fead0989f512520d3f778410acffdabfa12cfe25dc7a
Jul 30 00:50:15.259601 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.259564    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"]
Jul 30 00:50:15.262939 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.262918    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:15.265542 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.265494    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 30 00:50:15.265542 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.265495    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 00:50:15.265711 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.265501    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 30 00:50:15.265711 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.265632    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 30 00:50:15.265711 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.265683    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-cd49j\""
Jul 30 00:50:15.267129 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.267110    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2469cd23-8223-4779-a4ac-4268d35322db-tls-certs\") pod \"model-serving-api-54d89bd5fc-hbwcm\" (UID: \"2469cd23-8223-4779-a4ac-4268d35322db\") " pod="opendatahub/model-serving-api-54d89bd5fc-hbwcm"
Jul 30 00:50:15.267206 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.267144    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg5w6\" (UniqueName: \"kubernetes.io/projected/2469cd23-8223-4779-a4ac-4268d35322db-kube-api-access-zg5w6\") pod \"model-serving-api-54d89bd5fc-hbwcm\" (UID: \"2469cd23-8223-4779-a4ac-4268d35322db\") " pod="opendatahub/model-serving-api-54d89bd5fc-hbwcm"
Jul 30 00:50:15.267268 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:50:15.267250    2546 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 30 00:50:15.267329 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:50:15.267317    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2469cd23-8223-4779-a4ac-4268d35322db-tls-certs podName:2469cd23-8223-4779-a4ac-4268d35322db nodeName:}" failed. No retries permitted until 2026-07-30 00:50:15.7672989 +0000 UTC m=+402.990032697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2469cd23-8223-4779-a4ac-4268d35322db-tls-certs") pod "model-serving-api-54d89bd5fc-hbwcm" (UID: "2469cd23-8223-4779-a4ac-4268d35322db") : secret "model-serving-api-tls" not found
Jul 30 00:50:15.274965 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.274944    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"]
Jul 30 00:50:15.283205 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.283180    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg5w6\" (UniqueName: \"kubernetes.io/projected/2469cd23-8223-4779-a4ac-4268d35322db-kube-api-access-zg5w6\") pod \"model-serving-api-54d89bd5fc-hbwcm\" (UID: \"2469cd23-8223-4779-a4ac-4268d35322db\") " pod="opendatahub/model-serving-api-54d89bd5fc-hbwcm"
Jul 30 00:50:15.368495 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.368460    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b1a3b59d-b2a0-44b9-976f-37f214ffc755-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-rq8zb\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:15.368691 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.368510    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1a3b59d-b2a0-44b9-976f-37f214ffc755-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-rq8zb\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:15.368691 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.368574    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1a3b59d-b2a0-44b9-976f-37f214ffc755-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-rq8zb\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:15.368691 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.368611    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b1a3b59d-b2a0-44b9-976f-37f214ffc755-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-rq8zb\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:15.368691 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.368633    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvpvv\" (UniqueName: \"kubernetes.io/projected/b1a3b59d-b2a0-44b9-976f-37f214ffc755-kube-api-access-tvpvv\") pod \"istiod-openshift-gateway-7b7995d5cd-rq8zb\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:15.368691 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.368654    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b1a3b59d-b2a0-44b9-976f-37f214ffc755-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-rq8zb\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:15.368850 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.368701    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b1a3b59d-b2a0-44b9-976f-37f214ffc755-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-rq8zb\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:15.469833 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.469799    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b1a3b59d-b2a0-44b9-976f-37f214ffc755-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-rq8zb\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:15.469989 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.469848    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b1a3b59d-b2a0-44b9-976f-37f214ffc755-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-rq8zb\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:15.469989 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.469881    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b1a3b59d-b2a0-44b9-976f-37f214ffc755-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-rq8zb\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:15.469989 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.469930    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1a3b59d-b2a0-44b9-976f-37f214ffc755-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-rq8zb\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:15.469989 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.469967    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1a3b59d-b2a0-44b9-976f-37f214ffc755-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-rq8zb\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:15.470182 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.470135    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b1a3b59d-b2a0-44b9-976f-37f214ffc755-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-rq8zb\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:15.470235 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.470178    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvpvv\" (UniqueName: \"kubernetes.io/projected/b1a3b59d-b2a0-44b9-976f-37f214ffc755-kube-api-access-tvpvv\") pod \"istiod-openshift-gateway-7b7995d5cd-rq8zb\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:15.470942 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.470912    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b1a3b59d-b2a0-44b9-976f-37f214ffc755-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-rq8zb\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:15.472316 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.472291    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1a3b59d-b2a0-44b9-976f-37f214ffc755-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-rq8zb\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:15.472417 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.472344    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b1a3b59d-b2a0-44b9-976f-37f214ffc755-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-rq8zb\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:15.472501 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.472481    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b1a3b59d-b2a0-44b9-976f-37f214ffc755-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-rq8zb\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:15.472564 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.472499    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b1a3b59d-b2a0-44b9-976f-37f214ffc755-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-rq8zb\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:15.477937 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.477917    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1a3b59d-b2a0-44b9-976f-37f214ffc755-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-rq8zb\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:15.478011 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.477966    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvpvv\" (UniqueName: \"kubernetes.io/projected/b1a3b59d-b2a0-44b9-976f-37f214ffc755-kube-api-access-tvpvv\") pod \"istiod-openshift-gateway-7b7995d5cd-rq8zb\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:15.572835 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.572797    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:15.711538 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.711490    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"]
Jul 30 00:50:15.714778 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:50:15.714747    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1a3b59d_b2a0_44b9_976f_37f214ffc755.slice/crio-1b2d2639ea56de3c5be07b641e857b719f41a2fe9eb7e46eabc636967cec4398 WatchSource:0}: Error finding container 1b2d2639ea56de3c5be07b641e857b719f41a2fe9eb7e46eabc636967cec4398: Status 404 returned error can't find the container with id 1b2d2639ea56de3c5be07b641e857b719f41a2fe9eb7e46eabc636967cec4398
Jul 30 00:50:15.772017 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.771929    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2469cd23-8223-4779-a4ac-4268d35322db-tls-certs\") pod \"model-serving-api-54d89bd5fc-hbwcm\" (UID: \"2469cd23-8223-4779-a4ac-4268d35322db\") " pod="opendatahub/model-serving-api-54d89bd5fc-hbwcm"
Jul 30 00:50:15.774430 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.774408    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2469cd23-8223-4779-a4ac-4268d35322db-tls-certs\") pod \"model-serving-api-54d89bd5fc-hbwcm\" (UID: \"2469cd23-8223-4779-a4ac-4268d35322db\") " pod="opendatahub/model-serving-api-54d89bd5fc-hbwcm"
Jul 30 00:50:15.959574 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:15.959539    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-hbwcm"
Jul 30 00:50:16.040762 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:16.040685    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-jt7jk" event={"ID":"7902a946-c204-49e0-a986-fd5b0db8011c","Type":"ContainerStarted","Data":"b2539cf40fcab80816b4fead0989f512520d3f778410acffdabfa12cfe25dc7a"}
Jul 30 00:50:16.042033 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:16.041986    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb" event={"ID":"b1a3b59d-b2a0-44b9-976f-37f214ffc755","Type":"ContainerStarted","Data":"1b2d2639ea56de3c5be07b641e857b719f41a2fe9eb7e46eabc636967cec4398"}
Jul 30 00:50:16.145003 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:16.144914    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hbwcm"]
Jul 30 00:50:16.149684 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:50:16.148897    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2469cd23_8223_4779_a4ac_4268d35322db.slice/crio-291a7f0c4ac3734da6928bcdca822c90e41a0603ab970c85c172897015fc2142 WatchSource:0}: Error finding container 291a7f0c4ac3734da6928bcdca822c90e41a0603ab970c85c172897015fc2142: Status 404 returned error can't find the container with id 291a7f0c4ac3734da6928bcdca822c90e41a0603ab970c85c172897015fc2142
Jul 30 00:50:17.059350 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:17.059305    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-hbwcm" event={"ID":"2469cd23-8223-4779-a4ac-4268d35322db","Type":"ContainerStarted","Data":"291a7f0c4ac3734da6928bcdca822c90e41a0603ab970c85c172897015fc2142"}
Jul 30 00:50:20.677556 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:20.677509    2546 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 30 00:50:20.677963 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:20.677586    2546 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 30 00:50:21.080380 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:21.080342    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-hbwcm" event={"ID":"2469cd23-8223-4779-a4ac-4268d35322db","Type":"ContainerStarted","Data":"1ab4972c1166f8e25846822b7d6ddf4f2c99b7c399aadab848708146e5837714"}
Jul 30 00:50:21.080584 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:21.080440    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-hbwcm"
Jul 30 00:50:21.081891 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:21.081862    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-95755dfb5-nf8t8" event={"ID":"8563107f-b595-483e-aa72-9a6be121a415","Type":"ContainerStarted","Data":"e7d3d142734b0c9634bcf86dc41eb6dd56eb85915f965dbe3e2ef7ee58e6bd64"}
Jul 30 00:50:21.083398 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:21.083373    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-jt7jk" event={"ID":"7902a946-c204-49e0-a986-fd5b0db8011c","Type":"ContainerStarted","Data":"d026dd8516a9f5c78865432045b8121f7609194f1a3013f8a7ff9fc3c261cd4a"}
Jul 30 00:50:21.083515 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:21.083439    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-jt7jk"
Jul 30 00:50:21.084917 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:21.084895    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb" event={"ID":"b1a3b59d-b2a0-44b9-976f-37f214ffc755","Type":"ContainerStarted","Data":"c936fcc6ee0d1d7e3923a234e8313dd93a1c0af8bf191f6056b898df42f3ff98"}
Jul 30 00:50:21.085136 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:21.085115    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:21.087068 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:21.087040    2546 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-rq8zb container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 00:50:21.087162 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:21.087096    2546 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb" podUID="b1a3b59d-b2a0-44b9-976f-37f214ffc755" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 00:50:21.102775 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:21.102719    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-hbwcm" podStartSLOduration=1.583694682 podStartE2EDuration="6.102702138s" totalImagesPullingTime="4.519007456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:50:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:50:21.101637438 +0000 UTC m=+408.324371257" watchObservedRunningTime="2026-07-30 00:50:21.102702138 +0000 UTC m=+408.325435957"
Jul 30 00:50:21.118183 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:21.118143    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-95755dfb5-nf8t8" podStartSLOduration=1.698896376 podStartE2EDuration="8.11813036s" totalImagesPullingTime="6.419233984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:50:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:50:21.117951453 +0000 UTC m=+408.340685293" watchObservedRunningTime="2026-07-30 00:50:21.11813036 +0000 UTC m=+408.340864167"
Jul 30 00:50:21.135383 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:21.135329    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-jt7jk" podStartSLOduration=1.6360497189999998 podStartE2EDuration="7.135310476s" totalImagesPullingTime="5.499260757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:50:21.133649615 +0000 UTC m=+408.356383433" watchObservedRunningTime="2026-07-30 00:50:21.135310476 +0000 UTC m=+408.358044295"
Jul 30 00:50:21.153816 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:21.153774    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb" podStartSLOduration=1.193234026 podStartE2EDuration="6.153756006s" totalImagesPullingTime="4.96052198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:50:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:50:21.152487475 +0000 UTC m=+408.375221314" watchObservedRunningTime="2026-07-30 00:50:21.153756006 +0000 UTC m=+408.376489825"
Jul 30 00:50:22.090291 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:22.090258    2546 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-rq8zb container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 00:50:22.090781 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:22.090313    2546 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb" podUID="b1a3b59d-b2a0-44b9-976f-37f214ffc755" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 00:50:24.923344 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:24.923309    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7c4q4"]
Jul 30 00:50:24.928006 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:24.927973    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7c4q4"
Jul 30 00:50:24.931648 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:24.931623    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 30 00:50:24.931796 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:24.931628    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-ppvqs\""
Jul 30 00:50:24.931796 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:24.931734    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7pmdr"]
Jul 30 00:50:24.934994 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:24.934970    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7pmdr"
Jul 30 00:50:24.938171 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:24.938143    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7c4q4"]
Jul 30 00:50:24.938794 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:24.938775    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 30 00:50:24.938917 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:24.938782    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-595b6\""
Jul 30 00:50:24.941172 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:24.941149    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg86m\" (UniqueName: \"kubernetes.io/projected/ea6158a9-c3f8-4cb1-a9c6-6425a2bf5bcf-kube-api-access-wg86m\") pod \"kserve-controller-manager-d6f4b65b8-7c4q4\" (UID: \"ea6158a9-c3f8-4cb1-a9c6-6425a2bf5bcf\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7c4q4"
Jul 30 00:50:24.941298 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:24.941279    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea6158a9-c3f8-4cb1-a9c6-6425a2bf5bcf-cert\") pod \"kserve-controller-manager-d6f4b65b8-7c4q4\" (UID: \"ea6158a9-c3f8-4cb1-a9c6-6425a2bf5bcf\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7c4q4"
Jul 30 00:50:24.941353 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:24.941322    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c69ed81-87c8-4800-8640-6e54b1267eeb-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7pmdr\" (UID: \"0c69ed81-87c8-4800-8640-6e54b1267eeb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7pmdr"
Jul 30 00:50:24.941353 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:24.941341    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x96gk\" (UniqueName: \"kubernetes.io/projected/0c69ed81-87c8-4800-8640-6e54b1267eeb-kube-api-access-x96gk\") pod \"llmisvc-controller-manager-777c6bd7d5-7pmdr\" (UID: \"0c69ed81-87c8-4800-8640-6e54b1267eeb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7pmdr"
Jul 30 00:50:24.953692 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:24.953658    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7pmdr"]
Jul 30 00:50:25.042053 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:25.042015    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea6158a9-c3f8-4cb1-a9c6-6425a2bf5bcf-cert\") pod \"kserve-controller-manager-d6f4b65b8-7c4q4\" (UID: \"ea6158a9-c3f8-4cb1-a9c6-6425a2bf5bcf\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7c4q4"
Jul 30 00:50:25.042257 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:25.042062    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c69ed81-87c8-4800-8640-6e54b1267eeb-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7pmdr\" (UID: \"0c69ed81-87c8-4800-8640-6e54b1267eeb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7pmdr"
Jul 30 00:50:25.042257 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:25.042082    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x96gk\" (UniqueName: \"kubernetes.io/projected/0c69ed81-87c8-4800-8640-6e54b1267eeb-kube-api-access-x96gk\") pod \"llmisvc-controller-manager-777c6bd7d5-7pmdr\" (UID: \"0c69ed81-87c8-4800-8640-6e54b1267eeb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7pmdr"
Jul 30 00:50:25.042257 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:50:25.042191    2546 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 30 00:50:25.042257 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:25.042216    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wg86m\" (UniqueName: \"kubernetes.io/projected/ea6158a9-c3f8-4cb1-a9c6-6425a2bf5bcf-kube-api-access-wg86m\") pod \"kserve-controller-manager-d6f4b65b8-7c4q4\" (UID: \"ea6158a9-c3f8-4cb1-a9c6-6425a2bf5bcf\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7c4q4"
Jul 30 00:50:25.042442 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:50:25.042276    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea6158a9-c3f8-4cb1-a9c6-6425a2bf5bcf-cert podName:ea6158a9-c3f8-4cb1-a9c6-6425a2bf5bcf nodeName:}" failed. No retries permitted until 2026-07-30 00:50:25.54225449 +0000 UTC m=+412.764988310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ea6158a9-c3f8-4cb1-a9c6-6425a2bf5bcf-cert") pod "kserve-controller-manager-d6f4b65b8-7c4q4" (UID: "ea6158a9-c3f8-4cb1-a9c6-6425a2bf5bcf") : secret "kserve-webhook-server-cert" not found
Jul 30 00:50:25.044567 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:25.044546    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c69ed81-87c8-4800-8640-6e54b1267eeb-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7pmdr\" (UID: \"0c69ed81-87c8-4800-8640-6e54b1267eeb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7pmdr"
Jul 30 00:50:25.057299 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:25.057273    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x96gk\" (UniqueName: \"kubernetes.io/projected/0c69ed81-87c8-4800-8640-6e54b1267eeb-kube-api-access-x96gk\") pod \"llmisvc-controller-manager-777c6bd7d5-7pmdr\" (UID: \"0c69ed81-87c8-4800-8640-6e54b1267eeb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7pmdr"
Jul 30 00:50:25.059605 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:25.059584    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg86m\" (UniqueName: \"kubernetes.io/projected/ea6158a9-c3f8-4cb1-a9c6-6425a2bf5bcf-kube-api-access-wg86m\") pod \"kserve-controller-manager-d6f4b65b8-7c4q4\" (UID: \"ea6158a9-c3f8-4cb1-a9c6-6425a2bf5bcf\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7c4q4"
Jul 30 00:50:25.091044 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:25.091016    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:25.250578 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:25.250538    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7pmdr"
Jul 30 00:50:25.380431 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:25.380407    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7pmdr"]
Jul 30 00:50:25.383021 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:50:25.382992    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0c69ed81_87c8_4800_8640_6e54b1267eeb.slice/crio-8dc717e0b465c62f7783a39ae65c9b933a4ed561b99337071a53889e50c3ddc9 WatchSource:0}: Error finding container 8dc717e0b465c62f7783a39ae65c9b933a4ed561b99337071a53889e50c3ddc9: Status 404 returned error can't find the container with id 8dc717e0b465c62f7783a39ae65c9b933a4ed561b99337071a53889e50c3ddc9
Jul 30 00:50:25.546251 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:25.546173    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea6158a9-c3f8-4cb1-a9c6-6425a2bf5bcf-cert\") pod \"kserve-controller-manager-d6f4b65b8-7c4q4\" (UID: \"ea6158a9-c3f8-4cb1-a9c6-6425a2bf5bcf\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7c4q4"
Jul 30 00:50:25.548584 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:25.548559    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea6158a9-c3f8-4cb1-a9c6-6425a2bf5bcf-cert\") pod \"kserve-controller-manager-d6f4b65b8-7c4q4\" (UID: \"ea6158a9-c3f8-4cb1-a9c6-6425a2bf5bcf\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7c4q4"
Jul 30 00:50:25.841391 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:25.841307    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7c4q4"
Jul 30 00:50:25.982230 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:25.982186    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7c4q4"]
Jul 30 00:50:25.984822 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:50:25.984793    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea6158a9_c3f8_4cb1_a9c6_6425a2bf5bcf.slice/crio-94ec07f5e5fa030e660633ad547a400f94939978c0b615c557c279509b591661 WatchSource:0}: Error finding container 94ec07f5e5fa030e660633ad547a400f94939978c0b615c557c279509b591661: Status 404 returned error can't find the container with id 94ec07f5e5fa030e660633ad547a400f94939978c0b615c557c279509b591661
Jul 30 00:50:26.108654 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:26.108567    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7c4q4" event={"ID":"ea6158a9-c3f8-4cb1-a9c6-6425a2bf5bcf","Type":"ContainerStarted","Data":"94ec07f5e5fa030e660633ad547a400f94939978c0b615c557c279509b591661"}
Jul 30 00:50:26.109985 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:26.109954    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7pmdr" event={"ID":"0c69ed81-87c8-4800-8640-6e54b1267eeb","Type":"ContainerStarted","Data":"8dc717e0b465c62f7783a39ae65c9b933a4ed561b99337071a53889e50c3ddc9"}
Jul 30 00:50:27.648185 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:27.648149    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"]
Jul 30 00:50:27.648636 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:27.648360    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb" podUID="b1a3b59d-b2a0-44b9-976f-37f214ffc755" containerName="discovery" containerID="cri-o://c936fcc6ee0d1d7e3923a234e8313dd93a1c0af8bf191f6056b898df42f3ff98" gracePeriod=30
Jul 30 00:50:28.090678 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:28.090638    2546 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-rq8zb container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.24:8080/ready\": dial tcp 10.133.0.24:8080: connect: connection refused" start-of-body=
Jul 30 00:50:28.090867 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:28.090705    2546 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb" podUID="b1a3b59d-b2a0-44b9-976f-37f214ffc755" containerName="discovery" probeResult="failure" output="Get \"http://10.133.0.24:8080/ready\": dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 30 00:50:28.673573 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:28.673545    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:28.776654 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:28.776563    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b1a3b59d-b2a0-44b9-976f-37f214ffc755-istio-csr-dns-cert\") pod \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") "
Jul 30 00:50:28.776654 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:28.776603    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1a3b59d-b2a0-44b9-976f-37f214ffc755-local-certs\") pod \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") "
Jul 30 00:50:28.776654 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:28.776642    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b1a3b59d-b2a0-44b9-976f-37f214ffc755-istio-csr-ca-configmap\") pod \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") "
Jul 30 00:50:28.776936 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:28.776685    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b1a3b59d-b2a0-44b9-976f-37f214ffc755-istio-kubeconfig\") pod \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") "
Jul 30 00:50:28.776936 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:28.776736    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1a3b59d-b2a0-44b9-976f-37f214ffc755-istio-token\") pod \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") "
Jul 30 00:50:28.776936 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:28.776760    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b1a3b59d-b2a0-44b9-976f-37f214ffc755-cacerts\") pod \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") "
Jul 30 00:50:28.776936 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:28.776814    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvpvv\" (UniqueName: \"kubernetes.io/projected/b1a3b59d-b2a0-44b9-976f-37f214ffc755-kube-api-access-tvpvv\") pod \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\" (UID: \"b1a3b59d-b2a0-44b9-976f-37f214ffc755\") "
Jul 30 00:50:28.777180 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:28.777143    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1a3b59d-b2a0-44b9-976f-37f214ffc755-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "b1a3b59d-b2a0-44b9-976f-37f214ffc755" (UID: "b1a3b59d-b2a0-44b9-976f-37f214ffc755"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 00:50:28.779233 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:28.779203    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1a3b59d-b2a0-44b9-976f-37f214ffc755-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "b1a3b59d-b2a0-44b9-976f-37f214ffc755" (UID: "b1a3b59d-b2a0-44b9-976f-37f214ffc755"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 00:50:28.779360 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:28.779300    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1a3b59d-b2a0-44b9-976f-37f214ffc755-istio-token" (OuterVolumeSpecName: "istio-token") pod "b1a3b59d-b2a0-44b9-976f-37f214ffc755" (UID: "b1a3b59d-b2a0-44b9-976f-37f214ffc755"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 00:50:28.779428 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:28.779367    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1a3b59d-b2a0-44b9-976f-37f214ffc755-local-certs" (OuterVolumeSpecName: "local-certs") pod "b1a3b59d-b2a0-44b9-976f-37f214ffc755" (UID: "b1a3b59d-b2a0-44b9-976f-37f214ffc755"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 00:50:28.779428 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:28.779369    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1a3b59d-b2a0-44b9-976f-37f214ffc755-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "b1a3b59d-b2a0-44b9-976f-37f214ffc755" (UID: "b1a3b59d-b2a0-44b9-976f-37f214ffc755"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 00:50:28.779560 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:28.779464    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1a3b59d-b2a0-44b9-976f-37f214ffc755-kube-api-access-tvpvv" (OuterVolumeSpecName: "kube-api-access-tvpvv") pod "b1a3b59d-b2a0-44b9-976f-37f214ffc755" (UID: "b1a3b59d-b2a0-44b9-976f-37f214ffc755"). InnerVolumeSpecName "kube-api-access-tvpvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 00:50:28.779630 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:28.779613    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1a3b59d-b2a0-44b9-976f-37f214ffc755-cacerts" (OuterVolumeSpecName: "cacerts") pod "b1a3b59d-b2a0-44b9-976f-37f214ffc755" (UID: "b1a3b59d-b2a0-44b9-976f-37f214ffc755"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 00:50:28.878237 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:28.878199    2546 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b1a3b59d-b2a0-44b9-976f-37f214ffc755-istio-kubeconfig\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:50:28.878237 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:28.878233    2546 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1a3b59d-b2a0-44b9-976f-37f214ffc755-istio-token\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:50:28.878237 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:28.878243    2546 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b1a3b59d-b2a0-44b9-976f-37f214ffc755-cacerts\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:50:28.878475 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:28.878251    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvpvv\" (UniqueName: \"kubernetes.io/projected/b1a3b59d-b2a0-44b9-976f-37f214ffc755-kube-api-access-tvpvv\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:50:28.878475 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:28.878263    2546 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b1a3b59d-b2a0-44b9-976f-37f214ffc755-istio-csr-dns-cert\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:50:28.878475 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:28.878277    2546 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1a3b59d-b2a0-44b9-976f-37f214ffc755-local-certs\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:50:28.878475 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:28.878286    2546 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b1a3b59d-b2a0-44b9-976f-37f214ffc755-istio-csr-ca-configmap\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:50:29.127439 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:29.127356    2546 generic.go:356] "Generic (PLEG): container finished" podID="b1a3b59d-b2a0-44b9-976f-37f214ffc755" containerID="c936fcc6ee0d1d7e3923a234e8313dd93a1c0af8bf191f6056b898df42f3ff98" exitCode=0
Jul 30 00:50:29.127439 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:29.127413    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb" event={"ID":"b1a3b59d-b2a0-44b9-976f-37f214ffc755","Type":"ContainerDied","Data":"c936fcc6ee0d1d7e3923a234e8313dd93a1c0af8bf191f6056b898df42f3ff98"}
Jul 30 00:50:29.127439 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:29.127429    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"
Jul 30 00:50:29.127720 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:29.127502    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb" event={"ID":"b1a3b59d-b2a0-44b9-976f-37f214ffc755","Type":"ContainerDied","Data":"1b2d2639ea56de3c5be07b641e857b719f41a2fe9eb7e46eabc636967cec4398"}
Jul 30 00:50:29.127720 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:29.127548    2546 scope.go:117] "RemoveContainer" containerID="c936fcc6ee0d1d7e3923a234e8313dd93a1c0af8bf191f6056b898df42f3ff98"
Jul 30 00:50:29.149017 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:29.148995    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"]
Jul 30 00:50:29.153355 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:29.153330    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rq8zb"]
Jul 30 00:50:29.338368 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:29.338344    2546 scope.go:117] "RemoveContainer" containerID="c936fcc6ee0d1d7e3923a234e8313dd93a1c0af8bf191f6056b898df42f3ff98"
Jul 30 00:50:29.338709 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:50:29.338686    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c936fcc6ee0d1d7e3923a234e8313dd93a1c0af8bf191f6056b898df42f3ff98\": container with ID starting with c936fcc6ee0d1d7e3923a234e8313dd93a1c0af8bf191f6056b898df42f3ff98 not found: ID does not exist" containerID="c936fcc6ee0d1d7e3923a234e8313dd93a1c0af8bf191f6056b898df42f3ff98"
Jul 30 00:50:29.338780 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:29.338719    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c936fcc6ee0d1d7e3923a234e8313dd93a1c0af8bf191f6056b898df42f3ff98"} err="failed to get container status \"c936fcc6ee0d1d7e3923a234e8313dd93a1c0af8bf191f6056b898df42f3ff98\": rpc error: code = NotFound desc = could not find container \"c936fcc6ee0d1d7e3923a234e8313dd93a1c0af8bf191f6056b898df42f3ff98\": container with ID starting with c936fcc6ee0d1d7e3923a234e8313dd93a1c0af8bf191f6056b898df42f3ff98 not found: ID does not exist"
Jul 30 00:50:29.362204 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:29.362171    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1a3b59d-b2a0-44b9-976f-37f214ffc755" path="/var/lib/kubelet/pods/b1a3b59d-b2a0-44b9-976f-37f214ffc755/volumes"
Jul 30 00:50:30.133422 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:30.133378    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7pmdr" event={"ID":"0c69ed81-87c8-4800-8640-6e54b1267eeb","Type":"ContainerStarted","Data":"29092869b0f519f703898292d73e6867c25e5594579ebb5a50692243693f314b"}
Jul 30 00:50:30.133911 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:30.133546    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7pmdr"
Jul 30 00:50:30.134856 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:30.134833    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7c4q4" event={"ID":"ea6158a9-c3f8-4cb1-a9c6-6425a2bf5bcf","Type":"ContainerStarted","Data":"ec23aca5143133c7a50a36bb2eb3291d00074c102856d18d2ba3a108e17d9be0"}
Jul 30 00:50:30.134982 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:30.134969    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7c4q4"
Jul 30 00:50:30.151896 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:30.151838    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7pmdr" podStartSLOduration=2.164282329 podStartE2EDuration="6.151823211s" totalImagesPullingTime="3.987540882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:50:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:50:30.150247426 +0000 UTC m=+417.372981240" watchObservedRunningTime="2026-07-30 00:50:30.151823211 +0000 UTC m=+417.374557028"
Jul 30 00:50:30.168771 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:30.168699    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7c4q4" podStartSLOduration=3.463727039 podStartE2EDuration="6.168683061s" totalImagesPullingTime="2.704956022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:50:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:50:30.16670718 +0000 UTC m=+417.389440998" watchObservedRunningTime="2026-07-30 00:50:30.168683061 +0000 UTC m=+417.391416878"
Jul 30 00:50:32.091960 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:32.091931    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-jt7jk"
Jul 30 00:50:32.094334 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:50:32.094305    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-hbwcm"
Jul 30 00:51:01.140764 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:01.140677    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7pmdr"
Jul 30 00:51:01.144287 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:01.144262    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7c4q4"
Jul 30 00:51:14.704976 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:14.704939    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tp6kq"]
Jul 30 00:51:14.705380 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:14.705274    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1a3b59d-b2a0-44b9-976f-37f214ffc755" containerName="discovery"
Jul 30 00:51:14.705380 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:14.705285    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1a3b59d-b2a0-44b9-976f-37f214ffc755" containerName="discovery"
Jul 30 00:51:14.705380 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:14.705345    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1a3b59d-b2a0-44b9-976f-37f214ffc755" containerName="discovery"
Jul 30 00:51:14.713439 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:14.713404    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tp6kq"
Jul 30 00:51:14.716159 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:14.716124    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 30 00:51:14.717440 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:14.717414    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 30 00:51:14.717823 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:14.717464    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-b6ljj\""
Jul 30 00:51:14.720193 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:14.720163    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tp6kq"]
Jul 30 00:51:14.859613 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:14.859568    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp2b6\" (UniqueName: \"kubernetes.io/projected/a5b19a01-010b-486b-bd77-d757f9339aaf-kube-api-access-mp2b6\") pod \"kuadrant-operator-catalog-tp6kq\" (UID: \"a5b19a01-010b-486b-bd77-d757f9339aaf\") " pod="kuadrant-system/kuadrant-operator-catalog-tp6kq"
Jul 30 00:51:14.960885 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:14.960789    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp2b6\" (UniqueName: \"kubernetes.io/projected/a5b19a01-010b-486b-bd77-d757f9339aaf-kube-api-access-mp2b6\") pod \"kuadrant-operator-catalog-tp6kq\" (UID: \"a5b19a01-010b-486b-bd77-d757f9339aaf\") " pod="kuadrant-system/kuadrant-operator-catalog-tp6kq"
Jul 30 00:51:14.968813 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:14.968775    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp2b6\" (UniqueName: \"kubernetes.io/projected/a5b19a01-010b-486b-bd77-d757f9339aaf-kube-api-access-mp2b6\") pod \"kuadrant-operator-catalog-tp6kq\" (UID: \"a5b19a01-010b-486b-bd77-d757f9339aaf\") " pod="kuadrant-system/kuadrant-operator-catalog-tp6kq"
Jul 30 00:51:15.026235 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:15.026197    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tp6kq"
Jul 30 00:51:15.084237 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:15.084199    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tp6kq"]
Jul 30 00:51:15.163220 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:15.163194    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tp6kq"]
Jul 30 00:51:15.165380 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:51:15.165357    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5b19a01_010b_486b_bd77_d757f9339aaf.slice/crio-8ff57e9be90cd5d8d9ee4c083bf4922231259f0857e9ff71c3ab122ff3bcd420 WatchSource:0}: Error finding container 8ff57e9be90cd5d8d9ee4c083bf4922231259f0857e9ff71c3ab122ff3bcd420: Status 404 returned error can't find the container with id 8ff57e9be90cd5d8d9ee4c083bf4922231259f0857e9ff71c3ab122ff3bcd420
Jul 30 00:51:15.287640 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:15.287605    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-68bvq"]
Jul 30 00:51:15.292452 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:15.292430    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-68bvq"
Jul 30 00:51:15.297857 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:15.297828    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-68bvq"]
Jul 30 00:51:15.321403 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:15.321365    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tp6kq" event={"ID":"a5b19a01-010b-486b-bd77-d757f9339aaf","Type":"ContainerStarted","Data":"8ff57e9be90cd5d8d9ee4c083bf4922231259f0857e9ff71c3ab122ff3bcd420"}
Jul 30 00:51:15.465394 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:15.465351    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms5sz\" (UniqueName: \"kubernetes.io/projected/f3851584-c3a7-48ec-b3ab-f43becc55337-kube-api-access-ms5sz\") pod \"kuadrant-operator-catalog-68bvq\" (UID: \"f3851584-c3a7-48ec-b3ab-f43becc55337\") " pod="kuadrant-system/kuadrant-operator-catalog-68bvq"
Jul 30 00:51:15.566145 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:15.566041    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms5sz\" (UniqueName: \"kubernetes.io/projected/f3851584-c3a7-48ec-b3ab-f43becc55337-kube-api-access-ms5sz\") pod \"kuadrant-operator-catalog-68bvq\" (UID: \"f3851584-c3a7-48ec-b3ab-f43becc55337\") " pod="kuadrant-system/kuadrant-operator-catalog-68bvq"
Jul 30 00:51:15.579456 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:15.579425    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms5sz\" (UniqueName: \"kubernetes.io/projected/f3851584-c3a7-48ec-b3ab-f43becc55337-kube-api-access-ms5sz\") pod \"kuadrant-operator-catalog-68bvq\" (UID: \"f3851584-c3a7-48ec-b3ab-f43becc55337\") " pod="kuadrant-system/kuadrant-operator-catalog-68bvq"
Jul 30 00:51:15.603566 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:15.603517    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-68bvq"
Jul 30 00:51:15.743088 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:15.743048    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-68bvq"]
Jul 30 00:51:15.834703 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:51:15.834616    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3851584_c3a7_48ec_b3ab_f43becc55337.slice/crio-f166277aa6b1416b79abe18d784b4d636daf334b1432d8f5e2614880d457bb9c WatchSource:0}: Error finding container f166277aa6b1416b79abe18d784b4d636daf334b1432d8f5e2614880d457bb9c: Status 404 returned error can't find the container with id f166277aa6b1416b79abe18d784b4d636daf334b1432d8f5e2614880d457bb9c
Jul 30 00:51:16.326790 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:16.326751    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-68bvq" event={"ID":"f3851584-c3a7-48ec-b3ab-f43becc55337","Type":"ContainerStarted","Data":"f166277aa6b1416b79abe18d784b4d636daf334b1432d8f5e2614880d457bb9c"}
Jul 30 00:51:18.337748 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:18.337712    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-68bvq" event={"ID":"f3851584-c3a7-48ec-b3ab-f43becc55337","Type":"ContainerStarted","Data":"7fcca4517603c71fa0e3d582058f15662367372d07f3a8cdfd71b4f1c014a6b2"}
Jul 30 00:51:18.339188 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:18.339161    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tp6kq" event={"ID":"a5b19a01-010b-486b-bd77-d757f9339aaf","Type":"ContainerStarted","Data":"7680f40a55ed85a6be01a4232c1cd339fa167864882eb0f0887075c099179eaa"}
Jul 30 00:51:18.339304 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:18.339189    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-tp6kq" podUID="a5b19a01-010b-486b-bd77-d757f9339aaf" containerName="registry-server" containerID="cri-o://7680f40a55ed85a6be01a4232c1cd339fa167864882eb0f0887075c099179eaa" gracePeriod=2
Jul 30 00:51:18.354061 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:18.354006    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-68bvq" podStartSLOduration=1.542689047 podStartE2EDuration="3.353985655s" totalImagesPullingTime="1.811296608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:51:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:51:18.351840986 +0000 UTC m=+465.574574804" watchObservedRunningTime="2026-07-30 00:51:18.353985655 +0000 UTC m=+465.576719474"
Jul 30 00:51:18.367059 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:18.367010    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-tp6kq" podStartSLOduration=1.882992617 podStartE2EDuration="4.36699375s" totalImagesPullingTime="2.484001133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:51:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:51:18.365756684 +0000 UTC m=+465.588490501" watchObservedRunningTime="2026-07-30 00:51:18.36699375 +0000 UTC m=+465.589727568"
Jul 30 00:51:18.588210 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:18.588155    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tp6kq"
Jul 30 00:51:18.693503 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:18.693465    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mp2b6\" (UniqueName: \"kubernetes.io/projected/a5b19a01-010b-486b-bd77-d757f9339aaf-kube-api-access-mp2b6\") pod \"a5b19a01-010b-486b-bd77-d757f9339aaf\" (UID: \"a5b19a01-010b-486b-bd77-d757f9339aaf\") "
Jul 30 00:51:18.695767 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:18.695738    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5b19a01-010b-486b-bd77-d757f9339aaf-kube-api-access-mp2b6" (OuterVolumeSpecName: "kube-api-access-mp2b6") pod "a5b19a01-010b-486b-bd77-d757f9339aaf" (UID: "a5b19a01-010b-486b-bd77-d757f9339aaf"). InnerVolumeSpecName "kube-api-access-mp2b6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 00:51:18.794356 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:18.794319    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mp2b6\" (UniqueName: \"kubernetes.io/projected/a5b19a01-010b-486b-bd77-d757f9339aaf-kube-api-access-mp2b6\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:51:19.344432 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:19.344397    2546 generic.go:356] "Generic (PLEG): container finished" podID="a5b19a01-010b-486b-bd77-d757f9339aaf" containerID="7680f40a55ed85a6be01a4232c1cd339fa167864882eb0f0887075c099179eaa" exitCode=0
Jul 30 00:51:19.344917 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:19.344442    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tp6kq"
Jul 30 00:51:19.344917 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:19.344481    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tp6kq" event={"ID":"a5b19a01-010b-486b-bd77-d757f9339aaf","Type":"ContainerDied","Data":"7680f40a55ed85a6be01a4232c1cd339fa167864882eb0f0887075c099179eaa"}
Jul 30 00:51:19.344917 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:19.344554    2546 scope.go:117] "RemoveContainer" containerID="7680f40a55ed85a6be01a4232c1cd339fa167864882eb0f0887075c099179eaa"
Jul 30 00:51:19.344917 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:19.344552    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tp6kq" event={"ID":"a5b19a01-010b-486b-bd77-d757f9339aaf","Type":"ContainerDied","Data":"8ff57e9be90cd5d8d9ee4c083bf4922231259f0857e9ff71c3ab122ff3bcd420"}
Jul 30 00:51:19.353194 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:19.353173    2546 scope.go:117] "RemoveContainer" containerID="7680f40a55ed85a6be01a4232c1cd339fa167864882eb0f0887075c099179eaa"
Jul 30 00:51:19.353446 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:51:19.353429    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7680f40a55ed85a6be01a4232c1cd339fa167864882eb0f0887075c099179eaa\": container with ID starting with 7680f40a55ed85a6be01a4232c1cd339fa167864882eb0f0887075c099179eaa not found: ID does not exist" containerID="7680f40a55ed85a6be01a4232c1cd339fa167864882eb0f0887075c099179eaa"
Jul 30 00:51:19.353493 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:19.353455    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7680f40a55ed85a6be01a4232c1cd339fa167864882eb0f0887075c099179eaa"} err="failed to get container status \"7680f40a55ed85a6be01a4232c1cd339fa167864882eb0f0887075c099179eaa\": rpc error: code = NotFound desc = could not find container \"7680f40a55ed85a6be01a4232c1cd339fa167864882eb0f0887075c099179eaa\": container with ID starting with 7680f40a55ed85a6be01a4232c1cd339fa167864882eb0f0887075c099179eaa not found: ID does not exist"
Jul 30 00:51:19.364853 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:19.364823    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tp6kq"]
Jul 30 00:51:19.366900 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:19.366877    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tp6kq"]
Jul 30 00:51:21.357569 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:21.357516    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5b19a01-010b-486b-bd77-d757f9339aaf" path="/var/lib/kubelet/pods/a5b19a01-010b-486b-bd77-d757f9339aaf/volumes"
Jul 30 00:51:25.603837 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:25.603789    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-68bvq"
Jul 30 00:51:25.604323 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:25.603851    2546 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-68bvq"
Jul 30 00:51:25.626138 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:25.626104    2546 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-68bvq"
Jul 30 00:51:26.397283 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:26.397255    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-68bvq"
Jul 30 00:51:34.029944 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.029908    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c85d6c89f-ftlbz"]
Jul 30 00:51:34.030339 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.030221    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5b19a01-010b-486b-bd77-d757f9339aaf" containerName="registry-server"
Jul 30 00:51:34.030339 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.030234    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5b19a01-010b-486b-bd77-d757f9339aaf" containerName="registry-server"
Jul 30 00:51:34.030339 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.030295    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5b19a01-010b-486b-bd77-d757f9339aaf" containerName="registry-server"
Jul 30 00:51:34.034401 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.034373    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:34.037022 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.036992    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 30 00:51:34.038141 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.038112    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 30 00:51:34.038263 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.038146    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 30 00:51:34.038263 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.038153    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 30 00:51:34.038263 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.038199    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wjdhw\""
Jul 30 00:51:34.038263 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.038204    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 30 00:51:34.038493 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.038478    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 30 00:51:34.038558 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.038495    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 30 00:51:34.042718 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.042697    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 30 00:51:34.045149 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.045055    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c85d6c89f-ftlbz"]
Jul 30 00:51:34.106863 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.106818    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plp94\" (UniqueName: \"kubernetes.io/projected/d29bb327-c2f4-4488-8e2e-f96d3bdaef42-kube-api-access-plp94\") pod \"console-c85d6c89f-ftlbz\" (UID: \"d29bb327-c2f4-4488-8e2e-f96d3bdaef42\") " pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:34.106863 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.106865    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d29bb327-c2f4-4488-8e2e-f96d3bdaef42-console-serving-cert\") pod \"console-c85d6c89f-ftlbz\" (UID: \"d29bb327-c2f4-4488-8e2e-f96d3bdaef42\") " pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:34.107098 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.106899    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d29bb327-c2f4-4488-8e2e-f96d3bdaef42-console-oauth-config\") pod \"console-c85d6c89f-ftlbz\" (UID: \"d29bb327-c2f4-4488-8e2e-f96d3bdaef42\") " pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:34.107098 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.106920    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d29bb327-c2f4-4488-8e2e-f96d3bdaef42-oauth-serving-cert\") pod \"console-c85d6c89f-ftlbz\" (UID: \"d29bb327-c2f4-4488-8e2e-f96d3bdaef42\") " pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:34.107098 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.106942    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d29bb327-c2f4-4488-8e2e-f96d3bdaef42-console-config\") pod \"console-c85d6c89f-ftlbz\" (UID: \"d29bb327-c2f4-4488-8e2e-f96d3bdaef42\") " pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:34.107098 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.106971    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d29bb327-c2f4-4488-8e2e-f96d3bdaef42-trusted-ca-bundle\") pod \"console-c85d6c89f-ftlbz\" (UID: \"d29bb327-c2f4-4488-8e2e-f96d3bdaef42\") " pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:34.107098 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.107053    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d29bb327-c2f4-4488-8e2e-f96d3bdaef42-service-ca\") pod \"console-c85d6c89f-ftlbz\" (UID: \"d29bb327-c2f4-4488-8e2e-f96d3bdaef42\") " pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:34.208306 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.208265    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d29bb327-c2f4-4488-8e2e-f96d3bdaef42-trusted-ca-bundle\") pod \"console-c85d6c89f-ftlbz\" (UID: \"d29bb327-c2f4-4488-8e2e-f96d3bdaef42\") " pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:34.208496 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.208340    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d29bb327-c2f4-4488-8e2e-f96d3bdaef42-service-ca\") pod \"console-c85d6c89f-ftlbz\" (UID: \"d29bb327-c2f4-4488-8e2e-f96d3bdaef42\") " pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:34.208496 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.208378    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plp94\" (UniqueName: \"kubernetes.io/projected/d29bb327-c2f4-4488-8e2e-f96d3bdaef42-kube-api-access-plp94\") pod \"console-c85d6c89f-ftlbz\" (UID: \"d29bb327-c2f4-4488-8e2e-f96d3bdaef42\") " pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:34.208496 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.208414    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d29bb327-c2f4-4488-8e2e-f96d3bdaef42-console-serving-cert\") pod \"console-c85d6c89f-ftlbz\" (UID: \"d29bb327-c2f4-4488-8e2e-f96d3bdaef42\") " pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:34.208496 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.208453    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d29bb327-c2f4-4488-8e2e-f96d3bdaef42-console-oauth-config\") pod \"console-c85d6c89f-ftlbz\" (UID: \"d29bb327-c2f4-4488-8e2e-f96d3bdaef42\") " pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:34.208496 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.208474    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d29bb327-c2f4-4488-8e2e-f96d3bdaef42-oauth-serving-cert\") pod \"console-c85d6c89f-ftlbz\" (UID: \"d29bb327-c2f4-4488-8e2e-f96d3bdaef42\") " pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:34.208786 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.208502    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d29bb327-c2f4-4488-8e2e-f96d3bdaef42-console-config\") pod \"console-c85d6c89f-ftlbz\" (UID: \"d29bb327-c2f4-4488-8e2e-f96d3bdaef42\") " pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:34.209225 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.209202    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d29bb327-c2f4-4488-8e2e-f96d3bdaef42-service-ca\") pod \"console-c85d6c89f-ftlbz\" (UID: \"d29bb327-c2f4-4488-8e2e-f96d3bdaef42\") " pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:34.209355 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.209327    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d29bb327-c2f4-4488-8e2e-f96d3bdaef42-oauth-serving-cert\") pod \"console-c85d6c89f-ftlbz\" (UID: \"d29bb327-c2f4-4488-8e2e-f96d3bdaef42\") " pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:34.209355 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.209336    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d29bb327-c2f4-4488-8e2e-f96d3bdaef42-console-config\") pod \"console-c85d6c89f-ftlbz\" (UID: \"d29bb327-c2f4-4488-8e2e-f96d3bdaef42\") " pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:34.209355 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.209341    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d29bb327-c2f4-4488-8e2e-f96d3bdaef42-trusted-ca-bundle\") pod \"console-c85d6c89f-ftlbz\" (UID: \"d29bb327-c2f4-4488-8e2e-f96d3bdaef42\") " pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:34.211575 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.211551    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d29bb327-c2f4-4488-8e2e-f96d3bdaef42-console-serving-cert\") pod \"console-c85d6c89f-ftlbz\" (UID: \"d29bb327-c2f4-4488-8e2e-f96d3bdaef42\") " pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:34.211685 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.211663    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d29bb327-c2f4-4488-8e2e-f96d3bdaef42-console-oauth-config\") pod \"console-c85d6c89f-ftlbz\" (UID: \"d29bb327-c2f4-4488-8e2e-f96d3bdaef42\") " pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:34.216668 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.216643    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plp94\" (UniqueName: \"kubernetes.io/projected/d29bb327-c2f4-4488-8e2e-f96d3bdaef42-kube-api-access-plp94\") pod \"console-c85d6c89f-ftlbz\" (UID: \"d29bb327-c2f4-4488-8e2e-f96d3bdaef42\") " pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:34.345773 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.345666    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:34.487884 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:34.487727    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c85d6c89f-ftlbz"]
Jul 30 00:51:34.490598 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:51:34.490559    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd29bb327_c2f4_4488_8e2e_f96d3bdaef42.slice/crio-7ce5077f20bbae9356a506e69671dd4a3c75f5dde585eaef7af89c245c165904 WatchSource:0}: Error finding container 7ce5077f20bbae9356a506e69671dd4a3c75f5dde585eaef7af89c245c165904: Status 404 returned error can't find the container with id 7ce5077f20bbae9356a506e69671dd4a3c75f5dde585eaef7af89c245c165904
Jul 30 00:51:35.416538 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:35.416484    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c85d6c89f-ftlbz" event={"ID":"d29bb327-c2f4-4488-8e2e-f96d3bdaef42","Type":"ContainerStarted","Data":"ad706ee31004de8ef407dd4a537c90d436d0532320a1b65b246ee7e6e7dc4f34"}
Jul 30 00:51:35.416912 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:35.416576    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c85d6c89f-ftlbz" event={"ID":"d29bb327-c2f4-4488-8e2e-f96d3bdaef42","Type":"ContainerStarted","Data":"7ce5077f20bbae9356a506e69671dd4a3c75f5dde585eaef7af89c245c165904"}
Jul 30 00:51:35.434181 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:35.434127    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c85d6c89f-ftlbz" podStartSLOduration=1.434111878 podStartE2EDuration="1.434111878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:51:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:51:35.432208013 +0000 UTC m=+482.654941830" watchObservedRunningTime="2026-07-30 00:51:35.434111878 +0000 UTC m=+482.656845696"
Jul 30 00:51:44.346683 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:44.346632    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:44.347075 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:44.346717    2546 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:44.351660 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:44.351635    2546 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:44.459682 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:44.459652    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c85d6c89f-ftlbz"
Jul 30 00:51:56.842910 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:56.842873    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"]
Jul 30 00:51:56.846116 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:56.846093    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"
Jul 30 00:51:56.848806 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:56.848783    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dgwdc\""
Jul 30 00:51:56.859640 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:56.859614    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"]
Jul 30 00:51:56.894868 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:56.894829    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd5rz\" (UniqueName: \"kubernetes.io/projected/ca7ef738-8859-4312-b493-0eeb3ec780d6-kube-api-access-wd5rz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p9r2w\" (UID: \"ca7ef738-8859-4312-b493-0eeb3ec780d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"
Jul 30 00:51:56.895026 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:56.894884    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca7ef738-8859-4312-b493-0eeb3ec780d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p9r2w\" (UID: \"ca7ef738-8859-4312-b493-0eeb3ec780d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"
Jul 30 00:51:56.995801 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:56.995765    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd5rz\" (UniqueName: \"kubernetes.io/projected/ca7ef738-8859-4312-b493-0eeb3ec780d6-kube-api-access-wd5rz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p9r2w\" (UID: \"ca7ef738-8859-4312-b493-0eeb3ec780d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"
Jul 30 00:51:56.995949 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:56.995823    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca7ef738-8859-4312-b493-0eeb3ec780d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p9r2w\" (UID: \"ca7ef738-8859-4312-b493-0eeb3ec780d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"
Jul 30 00:51:56.996224 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:56.996201    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca7ef738-8859-4312-b493-0eeb3ec780d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p9r2w\" (UID: \"ca7ef738-8859-4312-b493-0eeb3ec780d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"
Jul 30 00:51:57.004453 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:57.004434    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd5rz\" (UniqueName: \"kubernetes.io/projected/ca7ef738-8859-4312-b493-0eeb3ec780d6-kube-api-access-wd5rz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p9r2w\" (UID: \"ca7ef738-8859-4312-b493-0eeb3ec780d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"
Jul 30 00:51:57.156014 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:57.155936    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"
Jul 30 00:51:57.290384 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:57.290347    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"]
Jul 30 00:51:57.293034 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:51:57.293007    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca7ef738_8859_4312_b493_0eeb3ec780d6.slice/crio-0eab78bfe1c579b94cabd8f971ee52ead867ab2db530fb71ee5b39ac1fdb4a49 WatchSource:0}: Error finding container 0eab78bfe1c579b94cabd8f971ee52ead867ab2db530fb71ee5b39ac1fdb4a49: Status 404 returned error can't find the container with id 0eab78bfe1c579b94cabd8f971ee52ead867ab2db530fb71ee5b39ac1fdb4a49
Jul 30 00:51:57.516265 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:57.516234    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w" event={"ID":"ca7ef738-8859-4312-b493-0eeb3ec780d6","Type":"ContainerStarted","Data":"0eab78bfe1c579b94cabd8f971ee52ead867ab2db530fb71ee5b39ac1fdb4a49"}
Jul 30 00:51:58.844506 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:58.844469    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-nx2jl"]
Jul 30 00:51:58.847723 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:58.847694    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nx2jl"
Jul 30 00:51:58.850268 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:58.850234    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 30 00:51:58.850380 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:58.850237    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-nl7h9\""
Jul 30 00:51:58.857713 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:58.857685    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-nx2jl"]
Jul 30 00:51:58.912113 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:58.912079    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb4pg\" (UniqueName: \"kubernetes.io/projected/057120b0-8df7-4900-ad80-9d9bad5afbd1-kube-api-access-bb4pg\") pod \"dns-operator-controller-manager-864dd78c4c-nx2jl\" (UID: \"057120b0-8df7-4900-ad80-9d9bad5afbd1\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nx2jl"
Jul 30 00:51:59.013443 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:59.013406    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb4pg\" (UniqueName: \"kubernetes.io/projected/057120b0-8df7-4900-ad80-9d9bad5afbd1-kube-api-access-bb4pg\") pod \"dns-operator-controller-manager-864dd78c4c-nx2jl\" (UID: \"057120b0-8df7-4900-ad80-9d9bad5afbd1\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nx2jl"
Jul 30 00:51:59.024584 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:59.024553    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb4pg\" (UniqueName: \"kubernetes.io/projected/057120b0-8df7-4900-ad80-9d9bad5afbd1-kube-api-access-bb4pg\") pod \"dns-operator-controller-manager-864dd78c4c-nx2jl\" (UID: \"057120b0-8df7-4900-ad80-9d9bad5afbd1\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nx2jl"
Jul 30 00:51:59.161204 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:59.161123    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nx2jl"
Jul 30 00:51:59.307533 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:59.307488    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-nx2jl"]
Jul 30 00:51:59.309889 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:51:59.309855    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod057120b0_8df7_4900_ad80_9d9bad5afbd1.slice/crio-34aa5c3b90bee74af041cb711a3d8abf2ba9488a3bebfa588f2385f42ce24bf4 WatchSource:0}: Error finding container 34aa5c3b90bee74af041cb711a3d8abf2ba9488a3bebfa588f2385f42ce24bf4: Status 404 returned error can't find the container with id 34aa5c3b90bee74af041cb711a3d8abf2ba9488a3bebfa588f2385f42ce24bf4
Jul 30 00:51:59.526560 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:51:59.526505    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nx2jl" event={"ID":"057120b0-8df7-4900-ad80-9d9bad5afbd1","Type":"ContainerStarted","Data":"34aa5c3b90bee74af041cb711a3d8abf2ba9488a3bebfa588f2385f42ce24bf4"}
Jul 30 00:52:01.537807 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:01.537767    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nx2jl" event={"ID":"057120b0-8df7-4900-ad80-9d9bad5afbd1","Type":"ContainerStarted","Data":"99106a841e864bc276970b797241ad5811981de98006340c50decb5404ea54f1"}
Jul 30 00:52:01.538305 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:01.537880    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nx2jl"
Jul 30 00:52:01.554275 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:01.554222    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nx2jl" podStartSLOduration=1.879233759 podStartE2EDuration="3.554207526s" totalImagesPullingTime="1.674973767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:52:01.553493602 +0000 UTC m=+508.776227421" watchObservedRunningTime="2026-07-30 00:52:01.554207526 +0000 UTC m=+508.776941343"
Jul 30 00:52:02.132836 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:02.132792    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-7mhtx"]
Jul 30 00:52:02.136790 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:02.136764    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7mhtx"
Jul 30 00:52:02.139595 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:02.139566    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-jrrdn\""
Jul 30 00:52:02.146952 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:02.146925    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-7mhtx"]
Jul 30 00:52:02.244230 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:02.244192    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rn5g\" (UniqueName: \"kubernetes.io/projected/3e06937f-2029-46d1-90aa-17befcb538e1-kube-api-access-2rn5g\") pod \"authorino-operator-75cc8d5cb7-7mhtx\" (UID: \"3e06937f-2029-46d1-90aa-17befcb538e1\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-7mhtx"
Jul 30 00:52:02.345696 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:02.345653    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rn5g\" (UniqueName: \"kubernetes.io/projected/3e06937f-2029-46d1-90aa-17befcb538e1-kube-api-access-2rn5g\") pod \"authorino-operator-75cc8d5cb7-7mhtx\" (UID: \"3e06937f-2029-46d1-90aa-17befcb538e1\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-7mhtx"
Jul 30 00:52:02.385266 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:02.385180    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rn5g\" (UniqueName: \"kubernetes.io/projected/3e06937f-2029-46d1-90aa-17befcb538e1-kube-api-access-2rn5g\") pod \"authorino-operator-75cc8d5cb7-7mhtx\" (UID: \"3e06937f-2029-46d1-90aa-17befcb538e1\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-7mhtx"
Jul 30 00:52:02.448758 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:02.448722    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7mhtx"
Jul 30 00:52:02.971211 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:02.971184    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-7mhtx"]
Jul 30 00:52:02.973886 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:52:02.973860    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e06937f_2029_46d1_90aa_17befcb538e1.slice/crio-1f23afc959620c3ccea54d34fa18ce861f3a371fee1f925b0d646c9a79d23397 WatchSource:0}: Error finding container 1f23afc959620c3ccea54d34fa18ce861f3a371fee1f925b0d646c9a79d23397: Status 404 returned error can't find the container with id 1f23afc959620c3ccea54d34fa18ce861f3a371fee1f925b0d646c9a79d23397
Jul 30 00:52:03.548846 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:03.548811    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7mhtx" event={"ID":"3e06937f-2029-46d1-90aa-17befcb538e1","Type":"ContainerStarted","Data":"1f23afc959620c3ccea54d34fa18ce861f3a371fee1f925b0d646c9a79d23397"}
Jul 30 00:52:03.550318 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:03.550295    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w" event={"ID":"ca7ef738-8859-4312-b493-0eeb3ec780d6","Type":"ContainerStarted","Data":"b649bac3d1b1ca80b787874f52559ec52f25a8f1b57dc1bba0e95b55b71b740d"}
Jul 30 00:52:03.550429 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:03.550363    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"
Jul 30 00:52:03.570966 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:03.570914    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w" podStartSLOduration=1.9688971149999999 podStartE2EDuration="7.570875545s" totalImagesPullingTime="5.60197843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:51:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:52:03.568866178 +0000 UTC m=+510.791600012" watchObservedRunningTime="2026-07-30 00:52:03.570875545 +0000 UTC m=+510.793609362"
Jul 30 00:52:04.746895 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:04.746858    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"]
Jul 30 00:52:04.756552 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:04.756506    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"
Jul 30 00:52:04.759631 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:04.759606    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-mwdng\""
Jul 30 00:52:04.761885 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:04.761860    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"]
Jul 30 00:52:04.867500 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:04.867467    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4szv9\" (UniqueName: \"kubernetes.io/projected/6ee03e7a-d009-4842-82c8-2f4df89158e9-kube-api-access-4szv9\") pod \"limitador-operator-controller-manager-859d7b78bd-842nm\" (UID: \"6ee03e7a-d009-4842-82c8-2f4df89158e9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"
Jul 30 00:52:04.969132 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:04.968806    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4szv9\" (UniqueName: \"kubernetes.io/projected/6ee03e7a-d009-4842-82c8-2f4df89158e9-kube-api-access-4szv9\") pod \"limitador-operator-controller-manager-859d7b78bd-842nm\" (UID: \"6ee03e7a-d009-4842-82c8-2f4df89158e9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"
Jul 30 00:52:04.977508 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:04.977480    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4szv9\" (UniqueName: \"kubernetes.io/projected/6ee03e7a-d009-4842-82c8-2f4df89158e9-kube-api-access-4szv9\") pod \"limitador-operator-controller-manager-859d7b78bd-842nm\" (UID: \"6ee03e7a-d009-4842-82c8-2f4df89158e9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"
Jul 30 00:52:05.069923 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:05.069836    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"
Jul 30 00:52:05.409291 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:05.409260    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"]
Jul 30 00:52:05.411937 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:52:05.411909    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ee03e7a_d009_4842_82c8_2f4df89158e9.slice/crio-659b6fed8edaf8d25ad0c5c398266617895eb3aa90388911903514c7e95da32f WatchSource:0}: Error finding container 659b6fed8edaf8d25ad0c5c398266617895eb3aa90388911903514c7e95da32f: Status 404 returned error can't find the container with id 659b6fed8edaf8d25ad0c5c398266617895eb3aa90388911903514c7e95da32f
Jul 30 00:52:05.562373 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:05.562335    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm" event={"ID":"6ee03e7a-d009-4842-82c8-2f4df89158e9","Type":"ContainerStarted","Data":"659b6fed8edaf8d25ad0c5c398266617895eb3aa90388911903514c7e95da32f"}
Jul 30 00:52:05.563828 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:05.563802    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7mhtx" event={"ID":"3e06937f-2029-46d1-90aa-17befcb538e1","Type":"ContainerStarted","Data":"9a0b08e637bbc4e49ccd7d915b20f5ca5ed079b94d036a3e25e1a96dd25af067"}
Jul 30 00:52:05.564050 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:05.564034    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7mhtx"
Jul 30 00:52:05.584479 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:05.584394    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7mhtx" podStartSLOduration=1.231092446 podStartE2EDuration="3.584379939s" totalImagesPullingTime="2.353287493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:52:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:52:05.583100451 +0000 UTC m=+512.805834271" watchObservedRunningTime="2026-07-30 00:52:05.584379939 +0000 UTC m=+512.807113811"
Jul 30 00:52:07.576240 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:07.576204    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm" event={"ID":"6ee03e7a-d009-4842-82c8-2f4df89158e9","Type":"ContainerStarted","Data":"58779a4d1581d3ad33e6ee729b1a484097902b609249b52057dcecf1dfb78b32"}
Jul 30 00:52:07.576650 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:07.576285    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"
Jul 30 00:52:07.592301 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:07.592252    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm" podStartSLOduration=2.058501222 podStartE2EDuration="3.592237277s" totalImagesPullingTime="1.533736055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:52:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:52:07.591579981 +0000 UTC m=+514.814313799" watchObservedRunningTime="2026-07-30 00:52:07.592237277 +0000 UTC m=+514.814971137"
Jul 30 00:52:12.545930 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:12.545896    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nx2jl"
Jul 30 00:52:14.558978 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:14.558947    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"
Jul 30 00:52:16.128952 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.128915    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"]
Jul 30 00:52:16.129435 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.129095    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w" podUID="ca7ef738-8859-4312-b493-0eeb3ec780d6" containerName="manager" containerID="cri-o://b649bac3d1b1ca80b787874f52559ec52f25a8f1b57dc1bba0e95b55b71b740d" gracePeriod=2
Jul 30 00:52:16.137098 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.137068    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"]
Jul 30 00:52:16.139782 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.139714    2546 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"ca7ef738-8859-4312-b493-0eeb3ec780d6\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-p9r2w\": pods \"kuadrant-operator-controller-manager-8684c9ff49-p9r2w\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"
Jul 30 00:52:16.148245 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.148146    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"]
Jul 30 00:52:16.148833 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.148800    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm" podUID="6ee03e7a-d009-4842-82c8-2f4df89158e9" containerName="manager" containerID="cri-o://58779a4d1581d3ad33e6ee729b1a484097902b609249b52057dcecf1dfb78b32" gracePeriod=2
Jul 30 00:52:16.150775 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.150749    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"
Jul 30 00:52:16.150886 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.150808    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5wzjn"]
Jul 30 00:52:16.150886 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.150829    2546 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p9r2w\" is forbidden: User \"system:node:ip-10-0-141-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-85.ec2.internal' and this object" podUID="ca7ef738-8859-4312-b493-0eeb3ec780d6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"
Jul 30 00:52:16.151321 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.151302    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca7ef738-8859-4312-b493-0eeb3ec780d6" containerName="manager"
Jul 30 00:52:16.151384 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.151324    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca7ef738-8859-4312-b493-0eeb3ec780d6" containerName="manager"
Jul 30 00:52:16.151421 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.151403    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca7ef738-8859-4312-b493-0eeb3ec780d6" containerName="manager"
Jul 30 00:52:16.154609 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.154590    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5wzjn"
Jul 30 00:52:16.161877 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.161852    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"]
Jul 30 00:52:16.163187 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.163148    2546 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"6ee03e7a-d009-4842-82c8-2f4df89158e9\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-842nm\": pods \"limitador-operator-controller-manager-859d7b78bd-842nm\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"
Jul 30 00:52:16.169533 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.169494    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5wzjn"]
Jul 30 00:52:16.177933 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.177906    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x5dr5"]
Jul 30 00:52:16.178229 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.178215    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ee03e7a-d009-4842-82c8-2f4df89158e9" containerName="manager"
Jul 30 00:52:16.178229 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.178229    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ee03e7a-d009-4842-82c8-2f4df89158e9" containerName="manager"
Jul 30 00:52:16.178353 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.178294    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ee03e7a-d009-4842-82c8-2f4df89158e9" containerName="manager"
Jul 30 00:52:16.181315 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.181297    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x5dr5"
Jul 30 00:52:16.190600 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.190575    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x5dr5"]
Jul 30 00:52:16.196263 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.196234    2546 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p9r2w\" is forbidden: User \"system:node:ip-10-0-141-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-85.ec2.internal' and this object" podUID="ca7ef738-8859-4312-b493-0eeb3ec780d6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"
Jul 30 00:52:16.198604 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.198569    2546 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-842nm\" is forbidden: User \"system:node:ip-10-0-141-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-85.ec2.internal' and this object" podUID="6ee03e7a-d009-4842-82c8-2f4df89158e9" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"
Jul 30 00:52:16.223144 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.223112    2546 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p9r2w\" is forbidden: User \"system:node:ip-10-0-141-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-85.ec2.internal' and this object" podUID="ca7ef738-8859-4312-b493-0eeb3ec780d6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"
Jul 30 00:52:16.227247 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.227224    2546 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-842nm\" is forbidden: User \"system:node:ip-10-0-141-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-85.ec2.internal' and this object" podUID="6ee03e7a-d009-4842-82c8-2f4df89158e9" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"
Jul 30 00:52:16.268540 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.268498    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4nxq\" (UniqueName: \"kubernetes.io/projected/ffadb23e-f62b-4357-bf21-97825ddb0861-kube-api-access-m4nxq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5wzjn\" (UID: \"ffadb23e-f62b-4357-bf21-97825ddb0861\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5wzjn"
Jul 30 00:52:16.268669 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.268587    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6znnz\" (UniqueName: \"kubernetes.io/projected/392882c7-8ab1-49d1-8de2-510b564fee31-kube-api-access-6znnz\") pod \"limitador-operator-controller-manager-859d7b78bd-x5dr5\" (UID: \"392882c7-8ab1-49d1-8de2-510b564fee31\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x5dr5"
Jul 30 00:52:16.268669 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.268619    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ffadb23e-f62b-4357-bf21-97825ddb0861-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5wzjn\" (UID: \"ffadb23e-f62b-4357-bf21-97825ddb0861\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5wzjn"
Jul 30 00:52:16.369133 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.369107    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4nxq\" (UniqueName: \"kubernetes.io/projected/ffadb23e-f62b-4357-bf21-97825ddb0861-kube-api-access-m4nxq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5wzjn\" (UID: \"ffadb23e-f62b-4357-bf21-97825ddb0861\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5wzjn"
Jul 30 00:52:16.369260 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.369153    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6znnz\" (UniqueName: \"kubernetes.io/projected/392882c7-8ab1-49d1-8de2-510b564fee31-kube-api-access-6znnz\") pod \"limitador-operator-controller-manager-859d7b78bd-x5dr5\" (UID: \"392882c7-8ab1-49d1-8de2-510b564fee31\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x5dr5"
Jul 30 00:52:16.369260 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.369179    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ffadb23e-f62b-4357-bf21-97825ddb0861-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5wzjn\" (UID: \"ffadb23e-f62b-4357-bf21-97825ddb0861\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5wzjn"
Jul 30 00:52:16.369772 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.369752    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ffadb23e-f62b-4357-bf21-97825ddb0861-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5wzjn\" (UID: \"ffadb23e-f62b-4357-bf21-97825ddb0861\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5wzjn"
Jul 30 00:52:16.378000 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.377976    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6znnz\" (UniqueName: \"kubernetes.io/projected/392882c7-8ab1-49d1-8de2-510b564fee31-kube-api-access-6znnz\") pod \"limitador-operator-controller-manager-859d7b78bd-x5dr5\" (UID: \"392882c7-8ab1-49d1-8de2-510b564fee31\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x5dr5"
Jul 30 00:52:16.378218 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.378194    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4nxq\" (UniqueName: \"kubernetes.io/projected/ffadb23e-f62b-4357-bf21-97825ddb0861-kube-api-access-m4nxq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5wzjn\" (UID: \"ffadb23e-f62b-4357-bf21-97825ddb0861\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5wzjn"
Jul 30 00:52:16.400111 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.400089    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"
Jul 30 00:52:16.402185 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.402165    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"
Jul 30 00:52:16.402512 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.402491    2546 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p9r2w\" is forbidden: User \"system:node:ip-10-0-141-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-85.ec2.internal' and this object" podUID="ca7ef738-8859-4312-b493-0eeb3ec780d6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"
Jul 30 00:52:16.404482 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.404461    2546 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-842nm\" is forbidden: User \"system:node:ip-10-0-141-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-85.ec2.internal' and this object" podUID="6ee03e7a-d009-4842-82c8-2f4df89158e9" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"
Jul 30 00:52:16.406564 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.406544    2546 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p9r2w\" is forbidden: User \"system:node:ip-10-0-141-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-85.ec2.internal' and this object" podUID="ca7ef738-8859-4312-b493-0eeb3ec780d6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"
Jul 30 00:52:16.408546 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.408509    2546 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-842nm\" is forbidden: User \"system:node:ip-10-0-141-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-85.ec2.internal' and this object" podUID="6ee03e7a-d009-4842-82c8-2f4df89158e9" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"
Jul 30 00:52:16.570918 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.570882    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca7ef738-8859-4312-b493-0eeb3ec780d6-extensions-socket-volume\") pod \"ca7ef738-8859-4312-b493-0eeb3ec780d6\" (UID: \"ca7ef738-8859-4312-b493-0eeb3ec780d6\") "
Jul 30 00:52:16.570918 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.570925    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4szv9\" (UniqueName: \"kubernetes.io/projected/6ee03e7a-d009-4842-82c8-2f4df89158e9-kube-api-access-4szv9\") pod \"6ee03e7a-d009-4842-82c8-2f4df89158e9\" (UID: \"6ee03e7a-d009-4842-82c8-2f4df89158e9\") "
Jul 30 00:52:16.571159 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.570964    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wd5rz\" (UniqueName: \"kubernetes.io/projected/ca7ef738-8859-4312-b493-0eeb3ec780d6-kube-api-access-wd5rz\") pod \"ca7ef738-8859-4312-b493-0eeb3ec780d6\" (UID: \"ca7ef738-8859-4312-b493-0eeb3ec780d6\") "
Jul 30 00:52:16.571541 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.571493    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca7ef738-8859-4312-b493-0eeb3ec780d6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ca7ef738-8859-4312-b493-0eeb3ec780d6" (UID: "ca7ef738-8859-4312-b493-0eeb3ec780d6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 00:52:16.571762 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.571744    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7mhtx"
Jul 30 00:52:16.573127 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.573101    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca7ef738-8859-4312-b493-0eeb3ec780d6-kube-api-access-wd5rz" (OuterVolumeSpecName: "kube-api-access-wd5rz") pod "ca7ef738-8859-4312-b493-0eeb3ec780d6" (UID: "ca7ef738-8859-4312-b493-0eeb3ec780d6"). InnerVolumeSpecName "kube-api-access-wd5rz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 00:52:16.573218 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.573136    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ee03e7a-d009-4842-82c8-2f4df89158e9-kube-api-access-4szv9" (OuterVolumeSpecName: "kube-api-access-4szv9") pod "6ee03e7a-d009-4842-82c8-2f4df89158e9" (UID: "6ee03e7a-d009-4842-82c8-2f4df89158e9"). InnerVolumeSpecName "kube-api-access-4szv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 00:52:16.574373 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.574350    2546 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p9r2w\" is forbidden: User \"system:node:ip-10-0-141-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-85.ec2.internal' and this object" podUID="ca7ef738-8859-4312-b493-0eeb3ec780d6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"
Jul 30 00:52:16.576588 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.576565    2546 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-842nm\" is forbidden: User \"system:node:ip-10-0-141-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-85.ec2.internal' and this object" podUID="6ee03e7a-d009-4842-82c8-2f4df89158e9" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"
Jul 30 00:52:16.578691 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.578676    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5wzjn"
Jul 30 00:52:16.584727 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.584712    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x5dr5"
Jul 30 00:52:16.623290 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.623254    2546 generic.go:356] "Generic (PLEG): container finished" podID="6ee03e7a-d009-4842-82c8-2f4df89158e9" containerID="58779a4d1581d3ad33e6ee729b1a484097902b609249b52057dcecf1dfb78b32" exitCode=0
Jul 30 00:52:16.623487 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.623298    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"
Jul 30 00:52:16.623487 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.623428    2546 scope.go:117] "RemoveContainer" containerID="58779a4d1581d3ad33e6ee729b1a484097902b609249b52057dcecf1dfb78b32"
Jul 30 00:52:16.625347 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.625313    2546 generic.go:356] "Generic (PLEG): container finished" podID="ca7ef738-8859-4312-b493-0eeb3ec780d6" containerID="b649bac3d1b1ca80b787874f52559ec52f25a8f1b57dc1bba0e95b55b71b740d" exitCode=0
Jul 30 00:52:16.625504 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.625488    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"
Jul 30 00:52:16.626786 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.626738    2546 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p9r2w\" is forbidden: User \"system:node:ip-10-0-141-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-85.ec2.internal' and this object" podUID="ca7ef738-8859-4312-b493-0eeb3ec780d6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"
Jul 30 00:52:16.629993 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.629952    2546 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-842nm\" is forbidden: User \"system:node:ip-10-0-141-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-85.ec2.internal' and this object" podUID="6ee03e7a-d009-4842-82c8-2f4df89158e9" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"
Jul 30 00:52:16.632404 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.632302    2546 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p9r2w\" is forbidden: User \"system:node:ip-10-0-141-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-85.ec2.internal' and this object" podUID="ca7ef738-8859-4312-b493-0eeb3ec780d6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"
Jul 30 00:52:16.635201 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.634451    2546 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-842nm\" is forbidden: User \"system:node:ip-10-0-141-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-85.ec2.internal' and this object" podUID="6ee03e7a-d009-4842-82c8-2f4df89158e9" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"
Jul 30 00:52:16.637890 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.637872    2546 scope.go:117] "RemoveContainer" containerID="58779a4d1581d3ad33e6ee729b1a484097902b609249b52057dcecf1dfb78b32"
Jul 30 00:52:16.638216 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.638170    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58779a4d1581d3ad33e6ee729b1a484097902b609249b52057dcecf1dfb78b32\": container with ID starting with 58779a4d1581d3ad33e6ee729b1a484097902b609249b52057dcecf1dfb78b32 not found: ID does not exist" containerID="58779a4d1581d3ad33e6ee729b1a484097902b609249b52057dcecf1dfb78b32"
Jul 30 00:52:16.638301 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.638205    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58779a4d1581d3ad33e6ee729b1a484097902b609249b52057dcecf1dfb78b32"} err="failed to get container status \"58779a4d1581d3ad33e6ee729b1a484097902b609249b52057dcecf1dfb78b32\": rpc error: code = NotFound desc = could not find container \"58779a4d1581d3ad33e6ee729b1a484097902b609249b52057dcecf1dfb78b32\": container with ID starting with 58779a4d1581d3ad33e6ee729b1a484097902b609249b52057dcecf1dfb78b32 not found: ID does not exist"
Jul 30 00:52:16.638301 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.638232    2546 scope.go:117] "RemoveContainer" containerID="b649bac3d1b1ca80b787874f52559ec52f25a8f1b57dc1bba0e95b55b71b740d"
Jul 30 00:52:16.640388 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.640357    2546 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p9r2w\" is forbidden: User \"system:node:ip-10-0-141-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-85.ec2.internal' and this object" podUID="ca7ef738-8859-4312-b493-0eeb3ec780d6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"
Jul 30 00:52:16.642752 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.642375    2546 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-842nm\" is forbidden: User \"system:node:ip-10-0-141-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-85.ec2.internal' and this object" podUID="6ee03e7a-d009-4842-82c8-2f4df89158e9" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"
Jul 30 00:52:16.644742 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.644715    2546 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p9r2w\" is forbidden: User \"system:node:ip-10-0-141-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-85.ec2.internal' and this object" podUID="ca7ef738-8859-4312-b493-0eeb3ec780d6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"
Jul 30 00:52:16.648252 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.648221    2546 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-842nm\" is forbidden: User \"system:node:ip-10-0-141-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-85.ec2.internal' and this object" podUID="6ee03e7a-d009-4842-82c8-2f4df89158e9" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"
Jul 30 00:52:16.654491 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.653945    2546 scope.go:117] "RemoveContainer" containerID="b649bac3d1b1ca80b787874f52559ec52f25a8f1b57dc1bba0e95b55b71b740d"
Jul 30 00:52:16.654491 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.654297    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b649bac3d1b1ca80b787874f52559ec52f25a8f1b57dc1bba0e95b55b71b740d\": container with ID starting with b649bac3d1b1ca80b787874f52559ec52f25a8f1b57dc1bba0e95b55b71b740d not found: ID does not exist" containerID="b649bac3d1b1ca80b787874f52559ec52f25a8f1b57dc1bba0e95b55b71b740d"
Jul 30 00:52:16.654491 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.654327    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b649bac3d1b1ca80b787874f52559ec52f25a8f1b57dc1bba0e95b55b71b740d"} err="failed to get container status \"b649bac3d1b1ca80b787874f52559ec52f25a8f1b57dc1bba0e95b55b71b740d\": rpc error: code = NotFound desc = could not find container \"b649bac3d1b1ca80b787874f52559ec52f25a8f1b57dc1bba0e95b55b71b740d\": container with ID starting with b649bac3d1b1ca80b787874f52559ec52f25a8f1b57dc1bba0e95b55b71b740d not found: ID does not exist"
Jul 30 00:52:16.672076 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.672047    2546 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca7ef738-8859-4312-b493-0eeb3ec780d6-extensions-socket-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:52:16.672076 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.672075    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4szv9\" (UniqueName: \"kubernetes.io/projected/6ee03e7a-d009-4842-82c8-2f4df89158e9-kube-api-access-4szv9\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:52:16.672255 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.672089    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wd5rz\" (UniqueName: \"kubernetes.io/projected/ca7ef738-8859-4312-b493-0eeb3ec780d6-kube-api-access-wd5rz\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:52:16.734949 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.734923    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x5dr5"]
Jul 30 00:52:16.737892 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:52:16.737849    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod392882c7_8ab1_49d1_8de2_510b564fee31.slice/crio-3038defb1e65d96dd3d8e3bb6846292a0ac3bec38158e9e60b1f68b373fd9458 WatchSource:0}: Error finding container 3038defb1e65d96dd3d8e3bb6846292a0ac3bec38158e9e60b1f68b373fd9458: Status 404 returned error can't find the container with id 3038defb1e65d96dd3d8e3bb6846292a0ac3bec38158e9e60b1f68b373fd9458
Jul 30 00:52:16.759475 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.759452    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5wzjn"]
Jul 30 00:52:16.760113 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:52:16.760081    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffadb23e_f62b_4357_bf21_97825ddb0861.slice/crio-c04d9fee826a8287b89e5392425bdae600b60df69b562d9226be150605cca43e WatchSource:0}: Error finding container c04d9fee826a8287b89e5392425bdae600b60df69b562d9226be150605cca43e: Status 404 returned error can't find the container with id c04d9fee826a8287b89e5392425bdae600b60df69b562d9226be150605cca43e
Jul 30 00:52:16.958559 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.958446    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t9jvr"]
Jul 30 00:52:16.962484 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.962456    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t9jvr"
Jul 30 00:52:16.975363 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:16.975333    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t9jvr"]
Jul 30 00:52:16.992922 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.992884    2546 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p9r2w\" is forbidden: User \"system:node:ip-10-0-141-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-85.ec2.internal' and this object" podUID="ca7ef738-8859-4312-b493-0eeb3ec780d6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p9r2w"
Jul 30 00:52:16.995551 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:16.995507    2546 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-842nm\" is forbidden: User \"system:node:ip-10-0-141-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-85.ec2.internal' and this object" podUID="6ee03e7a-d009-4842-82c8-2f4df89158e9" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-842nm"
Jul 30 00:52:17.075116 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:17.075080    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0ad9948-0052-4a70-854c-58ee88e5e545-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-t9jvr\" (UID: \"d0ad9948-0052-4a70-854c-58ee88e5e545\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t9jvr"
Jul 30 00:52:17.075116 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:17.075127    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tggrr\" (UniqueName: \"kubernetes.io/projected/d0ad9948-0052-4a70-854c-58ee88e5e545-kube-api-access-tggrr\") pod \"kuadrant-operator-controller-manager-86878c769f-t9jvr\" (UID: \"d0ad9948-0052-4a70-854c-58ee88e5e545\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t9jvr"
Jul 30 00:52:17.175997 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:17.175968    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0ad9948-0052-4a70-854c-58ee88e5e545-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-t9jvr\" (UID: \"d0ad9948-0052-4a70-854c-58ee88e5e545\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t9jvr"
Jul 30 00:52:17.176342 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:17.176002    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tggrr\" (UniqueName: \"kubernetes.io/projected/d0ad9948-0052-4a70-854c-58ee88e5e545-kube-api-access-tggrr\") pod \"kuadrant-operator-controller-manager-86878c769f-t9jvr\" (UID: \"d0ad9948-0052-4a70-854c-58ee88e5e545\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t9jvr"
Jul 30 00:52:17.176396 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:17.176339    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0ad9948-0052-4a70-854c-58ee88e5e545-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-t9jvr\" (UID: \"d0ad9948-0052-4a70-854c-58ee88e5e545\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t9jvr"
Jul 30 00:52:17.204029 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:17.204001    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tggrr\" (UniqueName: \"kubernetes.io/projected/d0ad9948-0052-4a70-854c-58ee88e5e545-kube-api-access-tggrr\") pod \"kuadrant-operator-controller-manager-86878c769f-t9jvr\" (UID: \"d0ad9948-0052-4a70-854c-58ee88e5e545\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t9jvr"
Jul 30 00:52:17.272898 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:17.272866    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t9jvr"
Jul 30 00:52:17.359864 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:17.359829    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ee03e7a-d009-4842-82c8-2f4df89158e9" path="/var/lib/kubelet/pods/6ee03e7a-d009-4842-82c8-2f4df89158e9/volumes"
Jul 30 00:52:17.360320 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:17.360299    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca7ef738-8859-4312-b493-0eeb3ec780d6" path="/var/lib/kubelet/pods/ca7ef738-8859-4312-b493-0eeb3ec780d6/volumes"
Jul 30 00:52:17.424810 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:17.424788    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t9jvr"]
Jul 30 00:52:17.426166 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:52:17.426143    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0ad9948_0052_4a70_854c_58ee88e5e545.slice/crio-237ce58c316e74e988bf96bcaf89c413b40a6dfe0d889464672205e7a8c74aff WatchSource:0}: Error finding container 237ce58c316e74e988bf96bcaf89c413b40a6dfe0d889464672205e7a8c74aff: Status 404 returned error can't find the container with id 237ce58c316e74e988bf96bcaf89c413b40a6dfe0d889464672205e7a8c74aff
Jul 30 00:52:17.632072 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:17.631991    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x5dr5" event={"ID":"392882c7-8ab1-49d1-8de2-510b564fee31","Type":"ContainerStarted","Data":"364070935bce1f67ece6d05032ba86171890f2e0044f8899d20f07b764855874"}
Jul 30 00:52:17.632072 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:17.632039    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x5dr5" event={"ID":"392882c7-8ab1-49d1-8de2-510b564fee31","Type":"ContainerStarted","Data":"3038defb1e65d96dd3d8e3bb6846292a0ac3bec38158e9e60b1f68b373fd9458"}
Jul 30 00:52:17.632270 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:17.632090    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x5dr5"
Jul 30 00:52:17.635416 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:17.635394    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t9jvr" event={"ID":"d0ad9948-0052-4a70-854c-58ee88e5e545","Type":"ContainerStarted","Data":"d1264833516c3341af9bfadf6d98e33c1fba74375f4b78a636a7986cfced726e"}
Jul 30 00:52:17.635550 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:17.635433    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t9jvr" event={"ID":"d0ad9948-0052-4a70-854c-58ee88e5e545","Type":"ContainerStarted","Data":"237ce58c316e74e988bf96bcaf89c413b40a6dfe0d889464672205e7a8c74aff"}
Jul 30 00:52:17.635550 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:17.635457    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t9jvr"
Jul 30 00:52:17.636941 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:17.636918    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5wzjn" event={"ID":"ffadb23e-f62b-4357-bf21-97825ddb0861","Type":"ContainerStarted","Data":"76686b6be4b74bd0d27faab1d97eed9bcdfb4824a4279edf0648a3d3885fc132"}
Jul 30 00:52:17.637038 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:17.636951    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5wzjn" event={"ID":"ffadb23e-f62b-4357-bf21-97825ddb0861","Type":"ContainerStarted","Data":"c04d9fee826a8287b89e5392425bdae600b60df69b562d9226be150605cca43e"}
Jul 30 00:52:17.637088 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:17.637038    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5wzjn"
Jul 30 00:52:17.648798 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:17.648760    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x5dr5" podStartSLOduration=1.648747893 podStartE2EDuration="1.648747893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:52:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:52:17.648035617 +0000 UTC m=+524.870769437" watchObservedRunningTime="2026-07-30 00:52:17.648747893 +0000 UTC m=+524.871481711"
Jul 30 00:52:17.668140 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:17.668101    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t9jvr" podStartSLOduration=1.668089621 podStartE2EDuration="1.668089621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:52:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:52:17.66715403 +0000 UTC m=+524.889887851" watchObservedRunningTime="2026-07-30 00:52:17.668089621 +0000 UTC m=+524.890823439"
Jul 30 00:52:17.687787 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:17.687744    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5wzjn" podStartSLOduration=1.687729282 podStartE2EDuration="1.687729282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:52:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:52:17.68489413 +0000 UTC m=+524.907627962" watchObservedRunningTime="2026-07-30 00:52:17.687729282 +0000 UTC m=+524.910463099"
Jul 30 00:52:28.643832 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:28.643741    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5wzjn"
Jul 30 00:52:28.643832 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:28.643832    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x5dr5"
Jul 30 00:52:28.644285 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:28.643877    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t9jvr"
Jul 30 00:52:28.726462 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:28.726430    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5wzjn"]
Jul 30 00:52:28.726700 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:28.726658    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5wzjn" podUID="ffadb23e-f62b-4357-bf21-97825ddb0861" containerName="manager" containerID="cri-o://76686b6be4b74bd0d27faab1d97eed9bcdfb4824a4279edf0648a3d3885fc132" gracePeriod=10
Jul 30 00:52:28.982748 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:28.982722    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5wzjn"
Jul 30 00:52:29.079128 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:29.079097    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4nxq\" (UniqueName: \"kubernetes.io/projected/ffadb23e-f62b-4357-bf21-97825ddb0861-kube-api-access-m4nxq\") pod \"ffadb23e-f62b-4357-bf21-97825ddb0861\" (UID: \"ffadb23e-f62b-4357-bf21-97825ddb0861\") "
Jul 30 00:52:29.079304 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:29.079138    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ffadb23e-f62b-4357-bf21-97825ddb0861-extensions-socket-volume\") pod \"ffadb23e-f62b-4357-bf21-97825ddb0861\" (UID: \"ffadb23e-f62b-4357-bf21-97825ddb0861\") "
Jul 30 00:52:29.079581 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:29.079555    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffadb23e-f62b-4357-bf21-97825ddb0861-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ffadb23e-f62b-4357-bf21-97825ddb0861" (UID: "ffadb23e-f62b-4357-bf21-97825ddb0861"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 00:52:29.081171 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:29.081150    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffadb23e-f62b-4357-bf21-97825ddb0861-kube-api-access-m4nxq" (OuterVolumeSpecName: "kube-api-access-m4nxq") pod "ffadb23e-f62b-4357-bf21-97825ddb0861" (UID: "ffadb23e-f62b-4357-bf21-97825ddb0861"). InnerVolumeSpecName "kube-api-access-m4nxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 00:52:29.179915 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:29.179842    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4nxq\" (UniqueName: \"kubernetes.io/projected/ffadb23e-f62b-4357-bf21-97825ddb0861-kube-api-access-m4nxq\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:52:29.179915 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:29.179867    2546 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ffadb23e-f62b-4357-bf21-97825ddb0861-extensions-socket-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:52:29.700719 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:29.700685    2546 generic.go:356] "Generic (PLEG): container finished" podID="ffadb23e-f62b-4357-bf21-97825ddb0861" containerID="76686b6be4b74bd0d27faab1d97eed9bcdfb4824a4279edf0648a3d3885fc132" exitCode=0
Jul 30 00:52:29.701113 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:29.700729    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5wzjn" event={"ID":"ffadb23e-f62b-4357-bf21-97825ddb0861","Type":"ContainerDied","Data":"76686b6be4b74bd0d27faab1d97eed9bcdfb4824a4279edf0648a3d3885fc132"}
Jul 30 00:52:29.701113 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:29.700743    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5wzjn"
Jul 30 00:52:29.701113 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:29.700772    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5wzjn" event={"ID":"ffadb23e-f62b-4357-bf21-97825ddb0861","Type":"ContainerDied","Data":"c04d9fee826a8287b89e5392425bdae600b60df69b562d9226be150605cca43e"}
Jul 30 00:52:29.701113 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:29.700794    2546 scope.go:117] "RemoveContainer" containerID="76686b6be4b74bd0d27faab1d97eed9bcdfb4824a4279edf0648a3d3885fc132"
Jul 30 00:52:29.710074 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:29.710050    2546 scope.go:117] "RemoveContainer" containerID="76686b6be4b74bd0d27faab1d97eed9bcdfb4824a4279edf0648a3d3885fc132"
Jul 30 00:52:29.710324 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:29.710304    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76686b6be4b74bd0d27faab1d97eed9bcdfb4824a4279edf0648a3d3885fc132\": container with ID starting with 76686b6be4b74bd0d27faab1d97eed9bcdfb4824a4279edf0648a3d3885fc132 not found: ID does not exist" containerID="76686b6be4b74bd0d27faab1d97eed9bcdfb4824a4279edf0648a3d3885fc132"
Jul 30 00:52:29.710374 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:29.710332    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76686b6be4b74bd0d27faab1d97eed9bcdfb4824a4279edf0648a3d3885fc132"} err="failed to get container status \"76686b6be4b74bd0d27faab1d97eed9bcdfb4824a4279edf0648a3d3885fc132\": rpc error: code = NotFound desc = could not find container \"76686b6be4b74bd0d27faab1d97eed9bcdfb4824a4279edf0648a3d3885fc132\": container with ID starting with 76686b6be4b74bd0d27faab1d97eed9bcdfb4824a4279edf0648a3d3885fc132 not found: ID does not exist"
Jul 30 00:52:29.720785 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:29.720754    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5wzjn"]
Jul 30 00:52:29.725405 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:29.725380    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5wzjn"]
Jul 30 00:52:31.357903 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:31.357869    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffadb23e-f62b-4357-bf21-97825ddb0861" path="/var/lib/kubelet/pods/ffadb23e-f62b-4357-bf21-97825ddb0861/volumes"
Jul 30 00:52:47.312556 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:47.312507    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-w44cg"]
Jul 30 00:52:47.313049 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:47.312878    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffadb23e-f62b-4357-bf21-97825ddb0861" containerName="manager"
Jul 30 00:52:47.313049 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:47.312890    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffadb23e-f62b-4357-bf21-97825ddb0861" containerName="manager"
Jul 30 00:52:47.313049 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:47.312950    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffadb23e-f62b-4357-bf21-97825ddb0861" containerName="manager"
Jul 30 00:52:47.317957 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:47.317932    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-w44cg"
Jul 30 00:52:47.320478 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:47.320451    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 30 00:52:47.320619 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:47.320484    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5t6b4\""
Jul 30 00:52:47.326439 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:47.326413    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-w44cg"]
Jul 30 00:52:47.409016 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:47.408980    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-w44cg"]
Jul 30 00:52:47.429773 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:47.429745    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjvkd\" (UniqueName: \"kubernetes.io/projected/77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d-kube-api-access-cjvkd\") pod \"limitador-limitador-64c94fd64f-w44cg\" (UID: \"77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-w44cg"
Jul 30 00:52:47.429924 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:47.429800    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d-config-file\") pod \"limitador-limitador-64c94fd64f-w44cg\" (UID: \"77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-w44cg"
Jul 30 00:52:47.531168 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:47.531135    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d-config-file\") pod \"limitador-limitador-64c94fd64f-w44cg\" (UID: \"77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-w44cg"
Jul 30 00:52:47.531360 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:47.531239    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjvkd\" (UniqueName: \"kubernetes.io/projected/77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d-kube-api-access-cjvkd\") pod \"limitador-limitador-64c94fd64f-w44cg\" (UID: \"77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-w44cg"
Jul 30 00:52:47.531839 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:47.531820    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d-config-file\") pod \"limitador-limitador-64c94fd64f-w44cg\" (UID: \"77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-w44cg"
Jul 30 00:52:47.543807 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:47.543775    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjvkd\" (UniqueName: \"kubernetes.io/projected/77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d-kube-api-access-cjvkd\") pod \"limitador-limitador-64c94fd64f-w44cg\" (UID: \"77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-w44cg"
Jul 30 00:52:47.629576 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:47.629482    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-w44cg"
Jul 30 00:52:47.776266 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:47.776237    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-w44cg"]
Jul 30 00:52:47.778819 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:52:47.778786    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77b8f5fd_9b2e_4f5c_a2e1_7b10268b005d.slice/crio-4ea81f8facb84fcd662eb5cc43abd820dd762a396b073c1feaf3af443d47febc WatchSource:0}: Error finding container 4ea81f8facb84fcd662eb5cc43abd820dd762a396b073c1feaf3af443d47febc: Status 404 returned error can't find the container with id 4ea81f8facb84fcd662eb5cc43abd820dd762a396b073c1feaf3af443d47febc
Jul 30 00:52:47.788952 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:47.788913    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-w44cg" event={"ID":"77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d","Type":"ContainerStarted","Data":"4ea81f8facb84fcd662eb5cc43abd820dd762a396b073c1feaf3af443d47febc"}
Jul 30 00:52:48.201643 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:48.201604    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-6vqw8"]
Jul 30 00:52:48.204921 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:48.204903    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6vqw8"
Jul 30 00:52:48.207222 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:48.207191    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-q2ck5\""
Jul 30 00:52:48.211598 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:48.211575    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6vqw8"]
Jul 30 00:52:48.337298 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:48.337272    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj78h\" (UniqueName: \"kubernetes.io/projected/0aa25bee-9c6d-4ed2-9428-8fcd4534316a-kube-api-access-tj78h\") pod \"authorino-58cd955675-6vqw8\" (UID: \"0aa25bee-9c6d-4ed2-9428-8fcd4534316a\") " pod="kuadrant-system/authorino-58cd955675-6vqw8"
Jul 30 00:52:48.437912 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:48.437884    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj78h\" (UniqueName: \"kubernetes.io/projected/0aa25bee-9c6d-4ed2-9428-8fcd4534316a-kube-api-access-tj78h\") pod \"authorino-58cd955675-6vqw8\" (UID: \"0aa25bee-9c6d-4ed2-9428-8fcd4534316a\") " pod="kuadrant-system/authorino-58cd955675-6vqw8"
Jul 30 00:52:48.448930 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:48.448901    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj78h\" (UniqueName: \"kubernetes.io/projected/0aa25bee-9c6d-4ed2-9428-8fcd4534316a-kube-api-access-tj78h\") pod \"authorino-58cd955675-6vqw8\" (UID: \"0aa25bee-9c6d-4ed2-9428-8fcd4534316a\") " pod="kuadrant-system/authorino-58cd955675-6vqw8"
Jul 30 00:52:48.515982 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:48.515945    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6vqw8"
Jul 30 00:52:48.676580 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:48.676553    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6vqw8"]
Jul 30 00:52:48.798278 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:48.798168    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6vqw8" event={"ID":"0aa25bee-9c6d-4ed2-9428-8fcd4534316a","Type":"ContainerStarted","Data":"6d70e00d200ae0068bf0ee80b2732de495ae71b425349a906c9f987dc7cc528c"}
Jul 30 00:52:50.810369 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:50.810318    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-w44cg" event={"ID":"77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d","Type":"ContainerStarted","Data":"7af91de15f131e88e389ba6237f826973d7e73bc8c4370b20d9773aadf92044e"}
Jul 30 00:52:50.810836 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:50.810479    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-w44cg"
Jul 30 00:52:50.827629 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:50.827578    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-w44cg" podStartSLOduration=0.966553296 podStartE2EDuration="3.827560693s" totalImagesPullingTime="2.861007397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:52:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:52:50.825774365 +0000 UTC m=+558.048508209" watchObservedRunningTime="2026-07-30 00:52:50.827560693 +0000 UTC m=+558.050294512"
Jul 30 00:52:52.821023 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:52.820989    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6vqw8" event={"ID":"0aa25bee-9c6d-4ed2-9428-8fcd4534316a","Type":"ContainerStarted","Data":"8ed00a41ae451248b0405473fe53f2d0d3cedba3680f79ae5d576e0f6d409d68"}
Jul 30 00:52:52.837654 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:52.837610    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-6vqw8" podStartSLOduration=0.909734687 podStartE2EDuration="4.837595761s" totalImagesPullingTime="3.927861074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:52:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:52:52.835733032 +0000 UTC m=+560.058466855" watchObservedRunningTime="2026-07-30 00:52:52.837595761 +0000 UTC m=+560.060329579"
Jul 30 00:52:53.864941 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:53.864911    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6vqw8"]
Jul 30 00:52:54.830482 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:54.830441    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-6vqw8" podUID="0aa25bee-9c6d-4ed2-9428-8fcd4534316a" containerName="authorino" containerID="cri-o://8ed00a41ae451248b0405473fe53f2d0d3cedba3680f79ae5d576e0f6d409d68" gracePeriod=30
Jul 30 00:52:55.081824 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:55.081770    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6vqw8"
Jul 30 00:52:55.196378 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:55.196350    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tj78h\" (UniqueName: \"kubernetes.io/projected/0aa25bee-9c6d-4ed2-9428-8fcd4534316a-kube-api-access-tj78h\") pod \"0aa25bee-9c6d-4ed2-9428-8fcd4534316a\" (UID: \"0aa25bee-9c6d-4ed2-9428-8fcd4534316a\") "
Jul 30 00:52:55.198285 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:55.198261    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aa25bee-9c6d-4ed2-9428-8fcd4534316a-kube-api-access-tj78h" (OuterVolumeSpecName: "kube-api-access-tj78h") pod "0aa25bee-9c6d-4ed2-9428-8fcd4534316a" (UID: "0aa25bee-9c6d-4ed2-9428-8fcd4534316a"). InnerVolumeSpecName "kube-api-access-tj78h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 00:52:55.297162 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:55.297138    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tj78h\" (UniqueName: \"kubernetes.io/projected/0aa25bee-9c6d-4ed2-9428-8fcd4534316a-kube-api-access-tj78h\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:52:55.836940 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:55.836907    2546 generic.go:356] "Generic (PLEG): container finished" podID="0aa25bee-9c6d-4ed2-9428-8fcd4534316a" containerID="8ed00a41ae451248b0405473fe53f2d0d3cedba3680f79ae5d576e0f6d409d68" exitCode=0
Jul 30 00:52:55.837155 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:55.836951    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6vqw8"
Jul 30 00:52:55.837155 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:55.836991    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6vqw8" event={"ID":"0aa25bee-9c6d-4ed2-9428-8fcd4534316a","Type":"ContainerDied","Data":"8ed00a41ae451248b0405473fe53f2d0d3cedba3680f79ae5d576e0f6d409d68"}
Jul 30 00:52:55.837155 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:55.837028    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6vqw8" event={"ID":"0aa25bee-9c6d-4ed2-9428-8fcd4534316a","Type":"ContainerDied","Data":"6d70e00d200ae0068bf0ee80b2732de495ae71b425349a906c9f987dc7cc528c"}
Jul 30 00:52:55.837155 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:55.837044    2546 scope.go:117] "RemoveContainer" containerID="8ed00a41ae451248b0405473fe53f2d0d3cedba3680f79ae5d576e0f6d409d68"
Jul 30 00:52:55.846159 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:55.846141    2546 scope.go:117] "RemoveContainer" containerID="8ed00a41ae451248b0405473fe53f2d0d3cedba3680f79ae5d576e0f6d409d68"
Jul 30 00:52:55.846437 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:52:55.846420    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ed00a41ae451248b0405473fe53f2d0d3cedba3680f79ae5d576e0f6d409d68\": container with ID starting with 8ed00a41ae451248b0405473fe53f2d0d3cedba3680f79ae5d576e0f6d409d68 not found: ID does not exist" containerID="8ed00a41ae451248b0405473fe53f2d0d3cedba3680f79ae5d576e0f6d409d68"
Jul 30 00:52:55.846481 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:55.846450    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ed00a41ae451248b0405473fe53f2d0d3cedba3680f79ae5d576e0f6d409d68"} err="failed to get container status \"8ed00a41ae451248b0405473fe53f2d0d3cedba3680f79ae5d576e0f6d409d68\": rpc error: code = NotFound desc = could not find container \"8ed00a41ae451248b0405473fe53f2d0d3cedba3680f79ae5d576e0f6d409d68\": container with ID starting with 8ed00a41ae451248b0405473fe53f2d0d3cedba3680f79ae5d576e0f6d409d68 not found: ID does not exist"
Jul 30 00:52:55.853319 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:55.853297    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6vqw8"]
Jul 30 00:52:55.857339 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:55.857321    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-6vqw8"]
Jul 30 00:52:57.358277 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:52:57.358243    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0aa25bee-9c6d-4ed2-9428-8fcd4534316a" path="/var/lib/kubelet/pods/0aa25bee-9c6d-4ed2-9428-8fcd4534316a/volumes"
Jul 30 00:53:01.816935 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:01.816897    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-w44cg"
Jul 30 00:53:01.876580 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:01.876541    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-w44cg"]
Jul 30 00:53:01.876750 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:01.876718    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-w44cg" podUID="77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d" containerName="limitador" containerID="cri-o://7af91de15f131e88e389ba6237f826973d7e73bc8c4370b20d9773aadf92044e" gracePeriod=30
Jul 30 00:53:02.423201 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:02.423174    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-w44cg"
Jul 30 00:53:02.558645 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:02.558612    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjvkd\" (UniqueName: \"kubernetes.io/projected/77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d-kube-api-access-cjvkd\") pod \"77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d\" (UID: \"77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d\") "
Jul 30 00:53:02.558835 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:02.558677    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d-config-file\") pod \"77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d\" (UID: \"77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d\") "
Jul 30 00:53:02.559043 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:02.559019    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d-config-file" (OuterVolumeSpecName: "config-file") pod "77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d" (UID: "77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 00:53:02.560714 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:02.560686    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d-kube-api-access-cjvkd" (OuterVolumeSpecName: "kube-api-access-cjvkd") pod "77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d" (UID: "77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d"). InnerVolumeSpecName "kube-api-access-cjvkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 00:53:02.660189 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:02.660154    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjvkd\" (UniqueName: \"kubernetes.io/projected/77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d-kube-api-access-cjvkd\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:53:02.660189 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:02.660183    2546 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d-config-file\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:53:02.870207 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:02.870120    2546 generic.go:356] "Generic (PLEG): container finished" podID="77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d" containerID="7af91de15f131e88e389ba6237f826973d7e73bc8c4370b20d9773aadf92044e" exitCode=0
Jul 30 00:53:02.870207 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:02.870160    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-w44cg"
Jul 30 00:53:02.870675 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:02.870213    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-w44cg" event={"ID":"77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d","Type":"ContainerDied","Data":"7af91de15f131e88e389ba6237f826973d7e73bc8c4370b20d9773aadf92044e"}
Jul 30 00:53:02.870675 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:02.870259    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-w44cg" event={"ID":"77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d","Type":"ContainerDied","Data":"4ea81f8facb84fcd662eb5cc43abd820dd762a396b073c1feaf3af443d47febc"}
Jul 30 00:53:02.870675 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:02.870284    2546 scope.go:117] "RemoveContainer" containerID="7af91de15f131e88e389ba6237f826973d7e73bc8c4370b20d9773aadf92044e"
Jul 30 00:53:02.878831 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:02.878628    2546 scope.go:117] "RemoveContainer" containerID="7af91de15f131e88e389ba6237f826973d7e73bc8c4370b20d9773aadf92044e"
Jul 30 00:53:02.878913 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:53:02.878895    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7af91de15f131e88e389ba6237f826973d7e73bc8c4370b20d9773aadf92044e\": container with ID starting with 7af91de15f131e88e389ba6237f826973d7e73bc8c4370b20d9773aadf92044e not found: ID does not exist" containerID="7af91de15f131e88e389ba6237f826973d7e73bc8c4370b20d9773aadf92044e"
Jul 30 00:53:02.878958 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:02.878922    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7af91de15f131e88e389ba6237f826973d7e73bc8c4370b20d9773aadf92044e"} err="failed to get container status \"7af91de15f131e88e389ba6237f826973d7e73bc8c4370b20d9773aadf92044e\": rpc error: code = NotFound desc = could not find container \"7af91de15f131e88e389ba6237f826973d7e73bc8c4370b20d9773aadf92044e\": container with ID starting with 7af91de15f131e88e389ba6237f826973d7e73bc8c4370b20d9773aadf92044e not found: ID does not exist"
Jul 30 00:53:02.891187 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:02.891167    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-w44cg"]
Jul 30 00:53:02.894708 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:02.894685    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-w44cg"]
Jul 30 00:53:03.358578 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:03.358542    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d" path="/var/lib/kubelet/pods/77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d/volumes"
Jul 30 00:53:03.825005 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:03.824973    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-x2brv"]
Jul 30 00:53:03.825345 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:03.825331    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aa25bee-9c6d-4ed2-9428-8fcd4534316a" containerName="authorino"
Jul 30 00:53:03.825399 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:03.825349    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aa25bee-9c6d-4ed2-9428-8fcd4534316a" containerName="authorino"
Jul 30 00:53:03.825399 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:03.825366    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d" containerName="limitador"
Jul 30 00:53:03.825399 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:03.825372    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d" containerName="limitador"
Jul 30 00:53:03.825492 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:03.825442    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aa25bee-9c6d-4ed2-9428-8fcd4534316a" containerName="authorino"
Jul 30 00:53:03.825492 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:03.825450    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="77b8f5fd-9b2e-4f5c-a2e1-7b10268b005d" containerName="limitador"
Jul 30 00:53:03.828075 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:03.828058    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x2brv"
Jul 30 00:53:03.830749 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:03.830721    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 30 00:53:03.830749 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:03.830739    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 00:53:03.830981 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:03.830760    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 30 00:53:03.831672 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:03.831652    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-862fn\""
Jul 30 00:53:03.835508 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:03.835481    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-x2brv"]
Jul 30 00:53:03.973890 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:03.973854    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfqqx\" (UniqueName: \"kubernetes.io/projected/fc0b6562-aad5-400e-bb56-fba4d2516b38-kube-api-access-gfqqx\") pod \"postgres-69fb4594cf-x2brv\" (UID: \"fc0b6562-aad5-400e-bb56-fba4d2516b38\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x2brv"
Jul 30 00:53:03.973890 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:03.973892    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fc0b6562-aad5-400e-bb56-fba4d2516b38-data\") pod \"postgres-69fb4594cf-x2brv\" (UID: \"fc0b6562-aad5-400e-bb56-fba4d2516b38\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x2brv"
Jul 30 00:53:04.075246 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:04.075157    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfqqx\" (UniqueName: \"kubernetes.io/projected/fc0b6562-aad5-400e-bb56-fba4d2516b38-kube-api-access-gfqqx\") pod \"postgres-69fb4594cf-x2brv\" (UID: \"fc0b6562-aad5-400e-bb56-fba4d2516b38\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x2brv"
Jul 30 00:53:04.075246 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:04.075197    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fc0b6562-aad5-400e-bb56-fba4d2516b38-data\") pod \"postgres-69fb4594cf-x2brv\" (UID: \"fc0b6562-aad5-400e-bb56-fba4d2516b38\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x2brv"
Jul 30 00:53:04.075603 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:04.075586    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fc0b6562-aad5-400e-bb56-fba4d2516b38-data\") pod \"postgres-69fb4594cf-x2brv\" (UID: \"fc0b6562-aad5-400e-bb56-fba4d2516b38\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x2brv"
Jul 30 00:53:04.083879 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:04.083837    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfqqx\" (UniqueName: \"kubernetes.io/projected/fc0b6562-aad5-400e-bb56-fba4d2516b38-kube-api-access-gfqqx\") pod \"postgres-69fb4594cf-x2brv\" (UID: \"fc0b6562-aad5-400e-bb56-fba4d2516b38\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x2brv"
Jul 30 00:53:04.139957 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:04.139920    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x2brv"
Jul 30 00:53:04.275462 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:04.275439    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-x2brv"]
Jul 30 00:53:04.277635 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:53:04.277607    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc0b6562_aad5_400e_bb56_fba4d2516b38.slice/crio-02dd68e39b5783555a8462b1912c8f18b8c7b9e7dd881df3696e04bc18e9ff90 WatchSource:0}: Error finding container 02dd68e39b5783555a8462b1912c8f18b8c7b9e7dd881df3696e04bc18e9ff90: Status 404 returned error can't find the container with id 02dd68e39b5783555a8462b1912c8f18b8c7b9e7dd881df3696e04bc18e9ff90
Jul 30 00:53:04.880898 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:04.880868    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x2brv" event={"ID":"fc0b6562-aad5-400e-bb56-fba4d2516b38","Type":"ContainerStarted","Data":"02dd68e39b5783555a8462b1912c8f18b8c7b9e7dd881df3696e04bc18e9ff90"}
Jul 30 00:53:09.911971 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:09.911932    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x2brv" event={"ID":"fc0b6562-aad5-400e-bb56-fba4d2516b38","Type":"ContainerStarted","Data":"7dc4c4a2f2a0c36c25326fb03df14b6c054d11645fd4484d3f97cc0470f91af3"}
Jul 30 00:53:09.912350 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:09.912028    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x2brv"
Jul 30 00:53:09.927901 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:09.927856    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x2brv" podStartSLOduration=2.1915134 podStartE2EDuration="6.927838746s" totalImagesPullingTime="4.736325346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:53:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:53:09.925547488 +0000 UTC m=+577.148281304" watchObservedRunningTime="2026-07-30 00:53:09.927838746 +0000 UTC m=+577.150572563"
Jul 30 00:53:15.943425 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:15.943391    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x2brv"
Jul 30 00:53:16.472426 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:16.472391    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xj77v"]
Jul 30 00:53:16.478499 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:16.478478    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xj77v"
Jul 30 00:53:16.481515 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:16.481488    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-q2ck5\""
Jul 30 00:53:16.483066 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:16.483043    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xj77v"]
Jul 30 00:53:16.588200 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:16.588169    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r6jk\" (UniqueName: \"kubernetes.io/projected/fb20a990-8cdd-4ca5-aaa2-e7bd579d2111-kube-api-access-4r6jk\") pod \"authorino-5d4b67c446-xj77v\" (UID: \"fb20a990-8cdd-4ca5-aaa2-e7bd579d2111\") " pod="kuadrant-system/authorino-5d4b67c446-xj77v"
Jul 30 00:53:16.689235 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:16.689202    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r6jk\" (UniqueName: \"kubernetes.io/projected/fb20a990-8cdd-4ca5-aaa2-e7bd579d2111-kube-api-access-4r6jk\") pod \"authorino-5d4b67c446-xj77v\" (UID: \"fb20a990-8cdd-4ca5-aaa2-e7bd579d2111\") " pod="kuadrant-system/authorino-5d4b67c446-xj77v"
Jul 30 00:53:16.697152 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:16.697125    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r6jk\" (UniqueName: \"kubernetes.io/projected/fb20a990-8cdd-4ca5-aaa2-e7bd579d2111-kube-api-access-4r6jk\") pod \"authorino-5d4b67c446-xj77v\" (UID: \"fb20a990-8cdd-4ca5-aaa2-e7bd579d2111\") " pod="kuadrant-system/authorino-5d4b67c446-xj77v"
Jul 30 00:53:16.704860 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:16.704831    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xj77v"]
Jul 30 00:53:16.705093 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:16.705078    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xj77v"
Jul 30 00:53:16.727845 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:16.727785    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-749b9874fd-rdx82"]
Jul 30 00:53:16.732992 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:16.732971    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-749b9874fd-rdx82"
Jul 30 00:53:16.737808 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:16.737786    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-749b9874fd-rdx82"]
Jul 30 00:53:16.813961 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:16.813931    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-749b9874fd-rdx82"]
Jul 30 00:53:16.814197 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:53:16.814177    2546 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-gh56k], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-749b9874fd-rdx82" podUID="5a496c5b-ceb8-4c4b-8abc-cbe6decd4b86"
Jul 30 00:53:16.834178 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:16.834157    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xj77v"]
Jul 30 00:53:16.836501 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:53:16.836473    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb20a990_8cdd_4ca5_aaa2_e7bd579d2111.slice/crio-b744765fd0d3192de7ec7283ec47b373503b35a3ba4ff11be325f158da47d919 WatchSource:0}: Error finding container b744765fd0d3192de7ec7283ec47b373503b35a3ba4ff11be325f158da47d919: Status 404 returned error can't find the container with id b744765fd0d3192de7ec7283ec47b373503b35a3ba4ff11be325f158da47d919
Jul 30 00:53:16.890533 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:16.890503    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh56k\" (UniqueName: \"kubernetes.io/projected/5a496c5b-ceb8-4c4b-8abc-cbe6decd4b86-kube-api-access-gh56k\") pod \"authorino-749b9874fd-rdx82\" (UID: \"5a496c5b-ceb8-4c4b-8abc-cbe6decd4b86\") " pod="kuadrant-system/authorino-749b9874fd-rdx82"
Jul 30 00:53:16.945786 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:16.945748    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-xj77v" event={"ID":"fb20a990-8cdd-4ca5-aaa2-e7bd579d2111","Type":"ContainerStarted","Data":"b744765fd0d3192de7ec7283ec47b373503b35a3ba4ff11be325f158da47d919"}
Jul 30 00:53:16.946171 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:16.945765    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-749b9874fd-rdx82"
Jul 30 00:53:16.953532 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:16.953493    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-749b9874fd-rdx82"
Jul 30 00:53:16.991394 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:16.991368    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh56k\" (UniqueName: \"kubernetes.io/projected/5a496c5b-ceb8-4c4b-8abc-cbe6decd4b86-kube-api-access-gh56k\") pod \"authorino-749b9874fd-rdx82\" (UID: \"5a496c5b-ceb8-4c4b-8abc-cbe6decd4b86\") " pod="kuadrant-system/authorino-749b9874fd-rdx82"
Jul 30 00:53:16.999571 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:16.999537    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh56k\" (UniqueName: \"kubernetes.io/projected/5a496c5b-ceb8-4c4b-8abc-cbe6decd4b86-kube-api-access-gh56k\") pod \"authorino-749b9874fd-rdx82\" (UID: \"5a496c5b-ceb8-4c4b-8abc-cbe6decd4b86\") " pod="kuadrant-system/authorino-749b9874fd-rdx82"
Jul 30 00:53:17.192181 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:17.192160    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gh56k\" (UniqueName: \"kubernetes.io/projected/5a496c5b-ceb8-4c4b-8abc-cbe6decd4b86-kube-api-access-gh56k\") pod \"5a496c5b-ceb8-4c4b-8abc-cbe6decd4b86\" (UID: \"5a496c5b-ceb8-4c4b-8abc-cbe6decd4b86\") "
Jul 30 00:53:17.194156 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:17.194131    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a496c5b-ceb8-4c4b-8abc-cbe6decd4b86-kube-api-access-gh56k" (OuterVolumeSpecName: "kube-api-access-gh56k") pod "5a496c5b-ceb8-4c4b-8abc-cbe6decd4b86" (UID: "5a496c5b-ceb8-4c4b-8abc-cbe6decd4b86"). InnerVolumeSpecName "kube-api-access-gh56k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 00:53:17.293905 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:17.293871    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gh56k\" (UniqueName: \"kubernetes.io/projected/5a496c5b-ceb8-4c4b-8abc-cbe6decd4b86-kube-api-access-gh56k\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:53:17.952759 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:17.952679    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-749b9874fd-rdx82"
Jul 30 00:53:17.953336 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:17.953015    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-xj77v" event={"ID":"fb20a990-8cdd-4ca5-aaa2-e7bd579d2111","Type":"ContainerStarted","Data":"cfc35a8f8c99e055e0939ed7c5e1d896658387f231b7d8a756efc551dd288d46"}
Jul 30 00:53:17.953422 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:17.953162    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-xj77v" podUID="fb20a990-8cdd-4ca5-aaa2-e7bd579d2111" containerName="authorino" containerID="cri-o://cfc35a8f8c99e055e0939ed7c5e1d896658387f231b7d8a756efc551dd288d46" gracePeriod=30
Jul 30 00:53:17.969638 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:17.969568    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-xj77v" podStartSLOduration=1.619809912 podStartE2EDuration="1.96954597s" totalImagesPullingTime="349.736058ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:53:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:53:17.967017572 +0000 UTC m=+585.189751391" watchObservedRunningTime="2026-07-30 00:53:17.96954597 +0000 UTC m=+585.192279787"
Jul 30 00:53:17.995944 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:17.995910    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-749b9874fd-rdx82"]
Jul 30 00:53:17.999662 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:17.999633    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-749b9874fd-rdx82"]
Jul 30 00:53:18.247849 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:18.247823    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xj77v"
Jul 30 00:53:18.402931 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:18.402898    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4r6jk\" (UniqueName: \"kubernetes.io/projected/fb20a990-8cdd-4ca5-aaa2-e7bd579d2111-kube-api-access-4r6jk\") pod \"fb20a990-8cdd-4ca5-aaa2-e7bd579d2111\" (UID: \"fb20a990-8cdd-4ca5-aaa2-e7bd579d2111\") "
Jul 30 00:53:18.404925 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:18.404903    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb20a990-8cdd-4ca5-aaa2-e7bd579d2111-kube-api-access-4r6jk" (OuterVolumeSpecName: "kube-api-access-4r6jk") pod "fb20a990-8cdd-4ca5-aaa2-e7bd579d2111" (UID: "fb20a990-8cdd-4ca5-aaa2-e7bd579d2111"). InnerVolumeSpecName "kube-api-access-4r6jk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 00:53:18.504251 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:18.504160    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r6jk\" (UniqueName: \"kubernetes.io/projected/fb20a990-8cdd-4ca5-aaa2-e7bd579d2111-kube-api-access-4r6jk\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:53:18.958936 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:18.958865    2546 generic.go:356] "Generic (PLEG): container finished" podID="fb20a990-8cdd-4ca5-aaa2-e7bd579d2111" containerID="cfc35a8f8c99e055e0939ed7c5e1d896658387f231b7d8a756efc551dd288d46" exitCode=0
Jul 30 00:53:18.958936 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:18.958903    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xj77v"
Jul 30 00:53:18.958936 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:18.958934    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-xj77v" event={"ID":"fb20a990-8cdd-4ca5-aaa2-e7bd579d2111","Type":"ContainerDied","Data":"cfc35a8f8c99e055e0939ed7c5e1d896658387f231b7d8a756efc551dd288d46"}
Jul 30 00:53:18.959458 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:18.958962    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-xj77v" event={"ID":"fb20a990-8cdd-4ca5-aaa2-e7bd579d2111","Type":"ContainerDied","Data":"b744765fd0d3192de7ec7283ec47b373503b35a3ba4ff11be325f158da47d919"}
Jul 30 00:53:18.959458 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:18.958979    2546 scope.go:117] "RemoveContainer" containerID="cfc35a8f8c99e055e0939ed7c5e1d896658387f231b7d8a756efc551dd288d46"
Jul 30 00:53:18.967440 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:18.967421    2546 scope.go:117] "RemoveContainer" containerID="cfc35a8f8c99e055e0939ed7c5e1d896658387f231b7d8a756efc551dd288d46"
Jul 30 00:53:18.967710 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:53:18.967694    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfc35a8f8c99e055e0939ed7c5e1d896658387f231b7d8a756efc551dd288d46\": container with ID starting with cfc35a8f8c99e055e0939ed7c5e1d896658387f231b7d8a756efc551dd288d46 not found: ID does not exist" containerID="cfc35a8f8c99e055e0939ed7c5e1d896658387f231b7d8a756efc551dd288d46"
Jul 30 00:53:18.967759 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:18.967717    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfc35a8f8c99e055e0939ed7c5e1d896658387f231b7d8a756efc551dd288d46"} err="failed to get container status \"cfc35a8f8c99e055e0939ed7c5e1d896658387f231b7d8a756efc551dd288d46\": rpc error: code = NotFound desc = could not find container \"cfc35a8f8c99e055e0939ed7c5e1d896658387f231b7d8a756efc551dd288d46\": container with ID starting with cfc35a8f8c99e055e0939ed7c5e1d896658387f231b7d8a756efc551dd288d46 not found: ID does not exist"
Jul 30 00:53:18.978625 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:18.978603    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xj77v"]
Jul 30 00:53:18.980827 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:18.980806    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xj77v"]
Jul 30 00:53:19.359712 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:19.359682    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a496c5b-ceb8-4c4b-8abc-cbe6decd4b86" path="/var/lib/kubelet/pods/5a496c5b-ceb8-4c4b-8abc-cbe6decd4b86/volumes"
Jul 30 00:53:19.359898 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:19.359887    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb20a990-8cdd-4ca5-aaa2-e7bd579d2111" path="/var/lib/kubelet/pods/fb20a990-8cdd-4ca5-aaa2-e7bd579d2111/volumes"
Jul 30 00:53:22.465919 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:22.465879    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c864b9fd9-q2lgt"]
Jul 30 00:53:22.466401 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:22.466369    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb20a990-8cdd-4ca5-aaa2-e7bd579d2111" containerName="authorino"
Jul 30 00:53:22.466401 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:22.466388    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb20a990-8cdd-4ca5-aaa2-e7bd579d2111" containerName="authorino"
Jul 30 00:53:22.466542 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:22.466512    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb20a990-8cdd-4ca5-aaa2-e7bd579d2111" containerName="authorino"
Jul 30 00:53:22.607249 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:22.607212    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c864b9fd9-q2lgt"]
Jul 30 00:53:22.607412 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:22.607342    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c864b9fd9-q2lgt"
Jul 30 00:53:22.610199 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:22.610181    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 30 00:53:22.610336 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:22.610201    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 30 00:53:22.610336 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:22.610201    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-q4wph\""
Jul 30 00:53:22.634389 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:22.634363    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj45k\" (UniqueName: \"kubernetes.io/projected/71fdf182-8f82-4c39-8648-d6b102a399b4-kube-api-access-pj45k\") pod \"maas-controller-6c864b9fd9-q2lgt\" (UID: \"71fdf182-8f82-4c39-8648-d6b102a399b4\") " pod="opendatahub/maas-controller-6c864b9fd9-q2lgt"
Jul 30 00:53:22.634560 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:22.634426    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71fdf182-8f82-4c39-8648-d6b102a399b4-webhook-cert\") pod \"maas-controller-6c864b9fd9-q2lgt\" (UID: \"71fdf182-8f82-4c39-8648-d6b102a399b4\") " pod="opendatahub/maas-controller-6c864b9fd9-q2lgt"
Jul 30 00:53:22.643235 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:22.643212    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-756cdf68f8-nv7bf"]
Jul 30 00:53:22.656440 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:22.656417    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-756cdf68f8-nv7bf"]
Jul 30 00:53:22.656585 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:22.656566    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-756cdf68f8-nv7bf"
Jul 30 00:53:22.735734 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:22.735699    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngvs6\" (UniqueName: \"kubernetes.io/projected/84fbcbec-79d9-42c4-a121-676adfab60e6-kube-api-access-ngvs6\") pod \"maas-controller-756cdf68f8-nv7bf\" (UID: \"84fbcbec-79d9-42c4-a121-676adfab60e6\") " pod="opendatahub/maas-controller-756cdf68f8-nv7bf"
Jul 30 00:53:22.735734 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:22.735738    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84fbcbec-79d9-42c4-a121-676adfab60e6-webhook-cert\") pod \"maas-controller-756cdf68f8-nv7bf\" (UID: \"84fbcbec-79d9-42c4-a121-676adfab60e6\") " pod="opendatahub/maas-controller-756cdf68f8-nv7bf"
Jul 30 00:53:22.735951 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:22.735833    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj45k\" (UniqueName: \"kubernetes.io/projected/71fdf182-8f82-4c39-8648-d6b102a399b4-kube-api-access-pj45k\") pod \"maas-controller-6c864b9fd9-q2lgt\" (UID: \"71fdf182-8f82-4c39-8648-d6b102a399b4\") " pod="opendatahub/maas-controller-6c864b9fd9-q2lgt"
Jul 30 00:53:22.735951 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:22.735908    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71fdf182-8f82-4c39-8648-d6b102a399b4-webhook-cert\") pod \"maas-controller-6c864b9fd9-q2lgt\" (UID: \"71fdf182-8f82-4c39-8648-d6b102a399b4\") " pod="opendatahub/maas-controller-6c864b9fd9-q2lgt"
Jul 30 00:53:22.736020 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:53:22.736008    2546 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 30 00:53:22.736087 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:53:22.736076    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71fdf182-8f82-4c39-8648-d6b102a399b4-webhook-cert podName:71fdf182-8f82-4c39-8648-d6b102a399b4 nodeName:}" failed. No retries permitted until 2026-07-30 00:53:23.236057463 +0000 UTC m=+590.458791265 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/71fdf182-8f82-4c39-8648-d6b102a399b4-webhook-cert") pod "maas-controller-6c864b9fd9-q2lgt" (UID: "71fdf182-8f82-4c39-8648-d6b102a399b4") : secret "maas-controller-webhook-cert" not found
Jul 30 00:53:22.744156 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:22.744134    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj45k\" (UniqueName: \"kubernetes.io/projected/71fdf182-8f82-4c39-8648-d6b102a399b4-kube-api-access-pj45k\") pod \"maas-controller-6c864b9fd9-q2lgt\" (UID: \"71fdf182-8f82-4c39-8648-d6b102a399b4\") " pod="opendatahub/maas-controller-6c864b9fd9-q2lgt"
Jul 30 00:53:22.836613 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:22.836581    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngvs6\" (UniqueName: \"kubernetes.io/projected/84fbcbec-79d9-42c4-a121-676adfab60e6-kube-api-access-ngvs6\") pod \"maas-controller-756cdf68f8-nv7bf\" (UID: \"84fbcbec-79d9-42c4-a121-676adfab60e6\") " pod="opendatahub/maas-controller-756cdf68f8-nv7bf"
Jul 30 00:53:22.836613 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:22.836620    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84fbcbec-79d9-42c4-a121-676adfab60e6-webhook-cert\") pod \"maas-controller-756cdf68f8-nv7bf\" (UID: \"84fbcbec-79d9-42c4-a121-676adfab60e6\") " pod="opendatahub/maas-controller-756cdf68f8-nv7bf"
Jul 30 00:53:22.839043 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:22.839011    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84fbcbec-79d9-42c4-a121-676adfab60e6-webhook-cert\") pod \"maas-controller-756cdf68f8-nv7bf\" (UID: \"84fbcbec-79d9-42c4-a121-676adfab60e6\") " pod="opendatahub/maas-controller-756cdf68f8-nv7bf"
Jul 30 00:53:22.844295 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:22.844272    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngvs6\" (UniqueName: \"kubernetes.io/projected/84fbcbec-79d9-42c4-a121-676adfab60e6-kube-api-access-ngvs6\") pod \"maas-controller-756cdf68f8-nv7bf\" (UID: \"84fbcbec-79d9-42c4-a121-676adfab60e6\") " pod="opendatahub/maas-controller-756cdf68f8-nv7bf"
Jul 30 00:53:22.967325 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:22.967292    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-756cdf68f8-nv7bf"
Jul 30 00:53:23.112195 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:23.112138    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-756cdf68f8-nv7bf"]
Jul 30 00:53:23.115019 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:53:23.114990    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84fbcbec_79d9_42c4_a121_676adfab60e6.slice/crio-79810100cee86f2366f4c4d09d101872fc80ebe73ef6ab6e4240013be5248f71 WatchSource:0}: Error finding container 79810100cee86f2366f4c4d09d101872fc80ebe73ef6ab6e4240013be5248f71: Status 404 returned error can't find the container with id 79810100cee86f2366f4c4d09d101872fc80ebe73ef6ab6e4240013be5248f71
Jul 30 00:53:23.239412 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:23.239377    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71fdf182-8f82-4c39-8648-d6b102a399b4-webhook-cert\") pod \"maas-controller-6c864b9fd9-q2lgt\" (UID: \"71fdf182-8f82-4c39-8648-d6b102a399b4\") " pod="opendatahub/maas-controller-6c864b9fd9-q2lgt"
Jul 30 00:53:23.241763 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:23.241717    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71fdf182-8f82-4c39-8648-d6b102a399b4-webhook-cert\") pod \"maas-controller-6c864b9fd9-q2lgt\" (UID: \"71fdf182-8f82-4c39-8648-d6b102a399b4\") " pod="opendatahub/maas-controller-6c864b9fd9-q2lgt"
Jul 30 00:53:23.517414 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:23.517327    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c864b9fd9-q2lgt"
Jul 30 00:53:23.647503 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:23.647481    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c864b9fd9-q2lgt"]
Jul 30 00:53:23.650117 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:53:23.650092    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71fdf182_8f82_4c39_8648_d6b102a399b4.slice/crio-2a4c1c55e098c558c737b2d455eb7f104da26d81eab2db4a096258e6946e5cd8 WatchSource:0}: Error finding container 2a4c1c55e098c558c737b2d455eb7f104da26d81eab2db4a096258e6946e5cd8: Status 404 returned error can't find the container with id 2a4c1c55e098c558c737b2d455eb7f104da26d81eab2db4a096258e6946e5cd8
Jul 30 00:53:23.984485 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:23.984455    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-756cdf68f8-nv7bf" event={"ID":"84fbcbec-79d9-42c4-a121-676adfab60e6","Type":"ContainerStarted","Data":"79810100cee86f2366f4c4d09d101872fc80ebe73ef6ab6e4240013be5248f71"}
Jul 30 00:53:23.985515 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:23.985491    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c864b9fd9-q2lgt" event={"ID":"71fdf182-8f82-4c39-8648-d6b102a399b4","Type":"ContainerStarted","Data":"2a4c1c55e098c558c737b2d455eb7f104da26d81eab2db4a096258e6946e5cd8"}
Jul 30 00:53:27.007874 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:27.007822    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c864b9fd9-q2lgt" event={"ID":"71fdf182-8f82-4c39-8648-d6b102a399b4","Type":"ContainerStarted","Data":"e70859bca21da4319fee1bb119f7127c953a88876677b9f9e3d0b09b1b6071a9"}
Jul 30 00:53:27.008370 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:27.008019    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c864b9fd9-q2lgt"
Jul 30 00:53:27.009924 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:27.009892    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-756cdf68f8-nv7bf" event={"ID":"84fbcbec-79d9-42c4-a121-676adfab60e6","Type":"ContainerStarted","Data":"2cd60f96364126ffcbc937385278a4a01440cac81dadfce200f5058a87b776f7"}
Jul 30 00:53:27.010047 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:27.009964    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-756cdf68f8-nv7bf"
Jul 30 00:53:27.026099 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:27.026038    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c864b9fd9-q2lgt" podStartSLOduration=2.414328715 podStartE2EDuration="5.026022585s" totalImagesPullingTime="2.61169387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:53:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:53:27.024865001 +0000 UTC m=+594.247598819" watchObservedRunningTime="2026-07-30 00:53:27.026022585 +0000 UTC m=+594.248756402"
Jul 30 00:53:27.041462 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:27.041364    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-756cdf68f8-nv7bf" podStartSLOduration=1.8882645359999999 podStartE2EDuration="5.041349092s" totalImagesPullingTime="3.153084556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:53:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:53:27.040069457 +0000 UTC m=+594.262803295" watchObservedRunningTime="2026-07-30 00:53:27.041349092 +0000 UTC m=+594.264082908"
Jul 30 00:53:33.189339 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:33.189312    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 00:53:33.190465 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:33.190444    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 00:53:38.018535 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:38.018486    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-756cdf68f8-nv7bf"
Jul 30 00:53:38.019077 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:38.018743    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c864b9fd9-q2lgt"
Jul 30 00:53:38.080994 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:38.080956    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c864b9fd9-q2lgt"]
Jul 30 00:53:38.081243 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:38.081215    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6c864b9fd9-q2lgt" podUID="71fdf182-8f82-4c39-8648-d6b102a399b4" containerName="manager" containerID="cri-o://e70859bca21da4319fee1bb119f7127c953a88876677b9f9e3d0b09b1b6071a9" gracePeriod=30
Jul 30 00:53:38.339059 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:38.339028    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c864b9fd9-q2lgt"
Jul 30 00:53:38.364274 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:38.364249    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71fdf182-8f82-4c39-8648-d6b102a399b4-webhook-cert\") pod \"71fdf182-8f82-4c39-8648-d6b102a399b4\" (UID: \"71fdf182-8f82-4c39-8648-d6b102a399b4\") "
Jul 30 00:53:38.364425 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:38.364354    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pj45k\" (UniqueName: \"kubernetes.io/projected/71fdf182-8f82-4c39-8648-d6b102a399b4-kube-api-access-pj45k\") pod \"71fdf182-8f82-4c39-8648-d6b102a399b4\" (UID: \"71fdf182-8f82-4c39-8648-d6b102a399b4\") "
Jul 30 00:53:38.366415 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:38.366387    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71fdf182-8f82-4c39-8648-d6b102a399b4-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "71fdf182-8f82-4c39-8648-d6b102a399b4" (UID: "71fdf182-8f82-4c39-8648-d6b102a399b4"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 00:53:38.366415 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:38.366400    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71fdf182-8f82-4c39-8648-d6b102a399b4-kube-api-access-pj45k" (OuterVolumeSpecName: "kube-api-access-pj45k") pod "71fdf182-8f82-4c39-8648-d6b102a399b4" (UID: "71fdf182-8f82-4c39-8648-d6b102a399b4"). InnerVolumeSpecName "kube-api-access-pj45k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 00:53:38.465679 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:38.465651    2546 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71fdf182-8f82-4c39-8648-d6b102a399b4-webhook-cert\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:53:38.465679 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:38.465676    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pj45k\" (UniqueName: \"kubernetes.io/projected/71fdf182-8f82-4c39-8648-d6b102a399b4-kube-api-access-pj45k\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:53:39.077103 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:39.077068    2546 generic.go:356] "Generic (PLEG): container finished" podID="71fdf182-8f82-4c39-8648-d6b102a399b4" containerID="e70859bca21da4319fee1bb119f7127c953a88876677b9f9e3d0b09b1b6071a9" exitCode=0
Jul 30 00:53:39.077612 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:39.077120    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c864b9fd9-q2lgt"
Jul 30 00:53:39.077612 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:39.077161    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c864b9fd9-q2lgt" event={"ID":"71fdf182-8f82-4c39-8648-d6b102a399b4","Type":"ContainerDied","Data":"e70859bca21da4319fee1bb119f7127c953a88876677b9f9e3d0b09b1b6071a9"}
Jul 30 00:53:39.077612 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:39.077221    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c864b9fd9-q2lgt" event={"ID":"71fdf182-8f82-4c39-8648-d6b102a399b4","Type":"ContainerDied","Data":"2a4c1c55e098c558c737b2d455eb7f104da26d81eab2db4a096258e6946e5cd8"}
Jul 30 00:53:39.077612 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:39.077258    2546 scope.go:117] "RemoveContainer" containerID="e70859bca21da4319fee1bb119f7127c953a88876677b9f9e3d0b09b1b6071a9"
Jul 30 00:53:39.086243 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:39.086048    2546 scope.go:117] "RemoveContainer" containerID="e70859bca21da4319fee1bb119f7127c953a88876677b9f9e3d0b09b1b6071a9"
Jul 30 00:53:39.086322 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:53:39.086304    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e70859bca21da4319fee1bb119f7127c953a88876677b9f9e3d0b09b1b6071a9\": container with ID starting with e70859bca21da4319fee1bb119f7127c953a88876677b9f9e3d0b09b1b6071a9 not found: ID does not exist" containerID="e70859bca21da4319fee1bb119f7127c953a88876677b9f9e3d0b09b1b6071a9"
Jul 30 00:53:39.086363 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:39.086331    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e70859bca21da4319fee1bb119f7127c953a88876677b9f9e3d0b09b1b6071a9"} err="failed to get container status \"e70859bca21da4319fee1bb119f7127c953a88876677b9f9e3d0b09b1b6071a9\": rpc error: code = NotFound desc = could not find container \"e70859bca21da4319fee1bb119f7127c953a88876677b9f9e3d0b09b1b6071a9\": container with ID starting with e70859bca21da4319fee1bb119f7127c953a88876677b9f9e3d0b09b1b6071a9 not found: ID does not exist"
Jul 30 00:53:39.100703 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:39.100673    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c864b9fd9-q2lgt"]
Jul 30 00:53:39.106118 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:39.106094    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6c864b9fd9-q2lgt"]
Jul 30 00:53:39.358497 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:39.358421    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71fdf182-8f82-4c39-8648-d6b102a399b4" path="/var/lib/kubelet/pods/71fdf182-8f82-4c39-8648-d6b102a399b4/volumes"
Jul 30 00:53:59.442482 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:59.442401    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6d87d4f98c-gccf6"]
Jul 30 00:53:59.442945 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:59.442782    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71fdf182-8f82-4c39-8648-d6b102a399b4" containerName="manager"
Jul 30 00:53:59.442945 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:59.442794    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="71fdf182-8f82-4c39-8648-d6b102a399b4" containerName="manager"
Jul 30 00:53:59.442945 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:59.442854    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="71fdf182-8f82-4c39-8648-d6b102a399b4" containerName="manager"
Jul 30 00:53:59.449986 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:59.449967    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d87d4f98c-gccf6"
Jul 30 00:53:59.453278 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:59.453002    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dnc2x\""
Jul 30 00:53:59.453278 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:59.453176    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 30 00:53:59.461004 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:59.460966    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d87d4f98c-gccf6"]
Jul 30 00:53:59.544584 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:59.544507    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/987f637e-b9af-48d1-b449-f0e95f618cb4-maas-api-tls\") pod \"maas-api-6d87d4f98c-gccf6\" (UID: \"987f637e-b9af-48d1-b449-f0e95f618cb4\") " pod="odh-ai-gateway-infra/maas-api-6d87d4f98c-gccf6"
Jul 30 00:53:59.544584 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:59.544582    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/987f637e-b9af-48d1-b449-f0e95f618cb4-openshift-service-ca\") pod \"maas-api-6d87d4f98c-gccf6\" (UID: \"987f637e-b9af-48d1-b449-f0e95f618cb4\") " pod="odh-ai-gateway-infra/maas-api-6d87d4f98c-gccf6"
Jul 30 00:53:59.544814 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:59.544619    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz5gd\" (UniqueName: \"kubernetes.io/projected/987f637e-b9af-48d1-b449-f0e95f618cb4-kube-api-access-bz5gd\") pod \"maas-api-6d87d4f98c-gccf6\" (UID: \"987f637e-b9af-48d1-b449-f0e95f618cb4\") " pod="odh-ai-gateway-infra/maas-api-6d87d4f98c-gccf6"
Jul 30 00:53:59.645592 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:59.645553    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/987f637e-b9af-48d1-b449-f0e95f618cb4-maas-api-tls\") pod \"maas-api-6d87d4f98c-gccf6\" (UID: \"987f637e-b9af-48d1-b449-f0e95f618cb4\") " pod="odh-ai-gateway-infra/maas-api-6d87d4f98c-gccf6"
Jul 30 00:53:59.645592 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:59.645593    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/987f637e-b9af-48d1-b449-f0e95f618cb4-openshift-service-ca\") pod \"maas-api-6d87d4f98c-gccf6\" (UID: \"987f637e-b9af-48d1-b449-f0e95f618cb4\") " pod="odh-ai-gateway-infra/maas-api-6d87d4f98c-gccf6"
Jul 30 00:53:59.645803 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:59.645733    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz5gd\" (UniqueName: \"kubernetes.io/projected/987f637e-b9af-48d1-b449-f0e95f618cb4-kube-api-access-bz5gd\") pod \"maas-api-6d87d4f98c-gccf6\" (UID: \"987f637e-b9af-48d1-b449-f0e95f618cb4\") " pod="odh-ai-gateway-infra/maas-api-6d87d4f98c-gccf6"
Jul 30 00:53:59.646191 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:59.646162    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/987f637e-b9af-48d1-b449-f0e95f618cb4-openshift-service-ca\") pod \"maas-api-6d87d4f98c-gccf6\" (UID: \"987f637e-b9af-48d1-b449-f0e95f618cb4\") " pod="odh-ai-gateway-infra/maas-api-6d87d4f98c-gccf6"
Jul 30 00:53:59.647950 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:59.647931    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/987f637e-b9af-48d1-b449-f0e95f618cb4-maas-api-tls\") pod \"maas-api-6d87d4f98c-gccf6\" (UID: \"987f637e-b9af-48d1-b449-f0e95f618cb4\") " pod="odh-ai-gateway-infra/maas-api-6d87d4f98c-gccf6"
Jul 30 00:53:59.656304 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:59.656273    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz5gd\" (UniqueName: \"kubernetes.io/projected/987f637e-b9af-48d1-b449-f0e95f618cb4-kube-api-access-bz5gd\") pod \"maas-api-6d87d4f98c-gccf6\" (UID: \"987f637e-b9af-48d1-b449-f0e95f618cb4\") " pod="odh-ai-gateway-infra/maas-api-6d87d4f98c-gccf6"
Jul 30 00:53:59.765440 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:59.765401    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d87d4f98c-gccf6"
Jul 30 00:53:59.912278 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:53:59.912193    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d87d4f98c-gccf6"]
Jul 30 00:53:59.914331 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:53:59.914293    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod987f637e_b9af_48d1_b449_f0e95f618cb4.slice/crio-db89eb0a585285499cc435204a464f9cebddb43c053b6bf0764d8eb87958ea1e WatchSource:0}: Error finding container db89eb0a585285499cc435204a464f9cebddb43c053b6bf0764d8eb87958ea1e: Status 404 returned error can't find the container with id db89eb0a585285499cc435204a464f9cebddb43c053b6bf0764d8eb87958ea1e
Jul 30 00:54:00.064746 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.064666    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-sfjl8"]
Jul 30 00:54:00.070590 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.070569    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sfjl8"
Jul 30 00:54:00.073309 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.073278    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 30 00:54:00.079932 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.079912    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-sfjl8"]
Jul 30 00:54:00.118282 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.118250    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-5692l"]
Jul 30 00:54:00.122281 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.122265    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-5692l"
Jul 30 00:54:00.124784 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.124765    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-h9fqh\""
Jul 30 00:54:00.133898 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.133874    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-5692l"]
Jul 30 00:54:00.150357 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.150328    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa8377ea-dafa-4afb-adfc-49a6ee20d360-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-sfjl8\" (UID: \"fa8377ea-dafa-4afb-adfc-49a6ee20d360\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sfjl8"
Jul 30 00:54:00.150491 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.150400    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvfdv\" (UniqueName: \"kubernetes.io/projected/fa8377ea-dafa-4afb-adfc-49a6ee20d360-kube-api-access-hvfdv\") pod \"payload-pre-processing-687dbb6cf6-sfjl8\" (UID: \"fa8377ea-dafa-4afb-adfc-49a6ee20d360\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sfjl8"
Jul 30 00:54:00.176171 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.176134    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d87d4f98c-gccf6" event={"ID":"987f637e-b9af-48d1-b449-f0e95f618cb4","Type":"ContainerStarted","Data":"db89eb0a585285499cc435204a464f9cebddb43c053b6bf0764d8eb87958ea1e"}
Jul 30 00:54:00.251678 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.251640    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c08d630-9e0a-412c-8531-968e388ff89e-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-5692l\" (UID: \"4c08d630-9e0a-412c-8531-968e388ff89e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-5692l"
Jul 30 00:54:00.251900 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.251703    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvfdv\" (UniqueName: \"kubernetes.io/projected/fa8377ea-dafa-4afb-adfc-49a6ee20d360-kube-api-access-hvfdv\") pod \"payload-pre-processing-687dbb6cf6-sfjl8\" (UID: \"fa8377ea-dafa-4afb-adfc-49a6ee20d360\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sfjl8"
Jul 30 00:54:00.251900 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.251788    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfvsd\" (UniqueName: \"kubernetes.io/projected/4c08d630-9e0a-412c-8531-968e388ff89e-kube-api-access-xfvsd\") pod \"payload-processing-5cc6cd444b-5692l\" (UID: \"4c08d630-9e0a-412c-8531-968e388ff89e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-5692l"
Jul 30 00:54:00.251900 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.251858    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa8377ea-dafa-4afb-adfc-49a6ee20d360-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-sfjl8\" (UID: \"fa8377ea-dafa-4afb-adfc-49a6ee20d360\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sfjl8"
Jul 30 00:54:00.252570 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.252516    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa8377ea-dafa-4afb-adfc-49a6ee20d360-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-sfjl8\" (UID: \"fa8377ea-dafa-4afb-adfc-49a6ee20d360\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sfjl8"
Jul 30 00:54:00.259817 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.259785    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvfdv\" (UniqueName: \"kubernetes.io/projected/fa8377ea-dafa-4afb-adfc-49a6ee20d360-kube-api-access-hvfdv\") pod \"payload-pre-processing-687dbb6cf6-sfjl8\" (UID: \"fa8377ea-dafa-4afb-adfc-49a6ee20d360\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sfjl8"
Jul 30 00:54:00.352697 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.352591    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfvsd\" (UniqueName: \"kubernetes.io/projected/4c08d630-9e0a-412c-8531-968e388ff89e-kube-api-access-xfvsd\") pod \"payload-processing-5cc6cd444b-5692l\" (UID: \"4c08d630-9e0a-412c-8531-968e388ff89e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-5692l"
Jul 30 00:54:00.352697 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.352677    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c08d630-9e0a-412c-8531-968e388ff89e-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-5692l\" (UID: \"4c08d630-9e0a-412c-8531-968e388ff89e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-5692l"
Jul 30 00:54:00.353436 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.353415    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c08d630-9e0a-412c-8531-968e388ff89e-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-5692l\" (UID: \"4c08d630-9e0a-412c-8531-968e388ff89e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-5692l"
Jul 30 00:54:00.360234 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.360208    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfvsd\" (UniqueName: \"kubernetes.io/projected/4c08d630-9e0a-412c-8531-968e388ff89e-kube-api-access-xfvsd\") pod \"payload-processing-5cc6cd444b-5692l\" (UID: \"4c08d630-9e0a-412c-8531-968e388ff89e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-5692l"
Jul 30 00:54:00.383062 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.383032    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sfjl8"
Jul 30 00:54:00.433057 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.433021    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-5692l"
Jul 30 00:54:00.533090 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.533063    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-sfjl8"]
Jul 30 00:54:00.536176 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:54:00.536018    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa8377ea_dafa_4afb_adfc_49a6ee20d360.slice/crio-e76107f48ec19034578b1bd2e26b4a1002aae934b964a6807dd56b28630d2c59 WatchSource:0}: Error finding container e76107f48ec19034578b1bd2e26b4a1002aae934b964a6807dd56b28630d2c59: Status 404 returned error can't find the container with id e76107f48ec19034578b1bd2e26b4a1002aae934b964a6807dd56b28630d2c59
Jul 30 00:54:00.588809 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:00.588764    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-5692l"]
Jul 30 00:54:00.591884 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:54:00.591851    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c08d630_9e0a_412c_8531_968e388ff89e.slice/crio-7823f0f6aeb637b43935673a7907a5da672c1a9e61b877278c00100fa7e34adf WatchSource:0}: Error finding container 7823f0f6aeb637b43935673a7907a5da672c1a9e61b877278c00100fa7e34adf: Status 404 returned error can't find the container with id 7823f0f6aeb637b43935673a7907a5da672c1a9e61b877278c00100fa7e34adf
Jul 30 00:54:01.185417 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:01.185329    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sfjl8" event={"ID":"fa8377ea-dafa-4afb-adfc-49a6ee20d360","Type":"ContainerStarted","Data":"e76107f48ec19034578b1bd2e26b4a1002aae934b964a6807dd56b28630d2c59"}
Jul 30 00:54:01.187251 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:01.187173    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-5692l" event={"ID":"4c08d630-9e0a-412c-8531-968e388ff89e","Type":"ContainerStarted","Data":"7823f0f6aeb637b43935673a7907a5da672c1a9e61b877278c00100fa7e34adf"}
Jul 30 00:54:03.201922 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:03.201834    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d87d4f98c-gccf6" event={"ID":"987f637e-b9af-48d1-b449-f0e95f618cb4","Type":"ContainerStarted","Data":"957ba1007461d7554afc2a8f7600a8152efeeb58ed5676bfade7a3bf0e841548"}
Jul 30 00:54:03.202371 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:03.201996    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6d87d4f98c-gccf6"
Jul 30 00:54:03.203500 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:03.203466    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sfjl8" event={"ID":"fa8377ea-dafa-4afb-adfc-49a6ee20d360","Type":"ContainerStarted","Data":"1f47bea808da87f0ce23416a52a4b2f9807b673b4e0f717d070347745d41972f"}
Jul 30 00:54:03.203642 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:03.203567    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sfjl8"
Jul 30 00:54:03.205149 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:03.205126    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-5692l" event={"ID":"4c08d630-9e0a-412c-8531-968e388ff89e","Type":"ContainerStarted","Data":"b2adc529a0f17ea4ed6ada2eeeb114ba3b939cc8f080ac5f625a150b85d4d543"}
Jul 30 00:54:03.205283 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:03.205248    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-5692l"
Jul 30 00:54:03.217771 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:03.217726    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6d87d4f98c-gccf6" podStartSLOduration=1.3290947659999999 podStartE2EDuration="4.217711495s" totalImagesPullingTime="2.888616729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:54:03.216888094 +0000 UTC m=+630.439621915" watchObservedRunningTime="2026-07-30 00:54:03.217711495 +0000 UTC m=+630.440445313"
Jul 30 00:54:03.232724 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:03.232673    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-5692l" podStartSLOduration=1.022511424 podStartE2EDuration="3.23265715s" totalImagesPullingTime="2.210145726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:54:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:54:03.230466348 +0000 UTC m=+630.453200167" watchObservedRunningTime="2026-07-30 00:54:03.23265715 +0000 UTC m=+630.455390967"
Jul 30 00:54:03.245555 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:03.245484    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sfjl8" podStartSLOduration=0.972980059 podStartE2EDuration="3.24546965s" totalImagesPullingTime="2.272489591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:54:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:54:03.243721518 +0000 UTC m=+630.466455337" watchObservedRunningTime="2026-07-30 00:54:03.24546965 +0000 UTC m=+630.468203469"
Jul 30 00:54:09.215361 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:09.215331    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6d87d4f98c-gccf6"
Jul 30 00:54:14.212324 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:14.212292    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-5692l"
Jul 30 00:54:14.212794 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:14.212377    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sfjl8"
Jul 30 00:54:24.341267 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.341232    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"]
Jul 30 00:54:24.344846 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.344819    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"
Jul 30 00:54:24.347400 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.347378    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 30 00:54:24.348591 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.348568    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-stf8p\""
Jul 30 00:54:24.348712 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.348572    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 30 00:54:24.348712 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.348572    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 30 00:54:24.354972 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.354948    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"]
Jul 30 00:54:24.368307 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.368276    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9\" (UID: \"026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"
Jul 30 00:54:24.368487 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.368320    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9\" (UID: \"026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"
Jul 30 00:54:24.368487 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.368427    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9\" (UID: \"026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"
Jul 30 00:54:24.368487 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.368471    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9\" (UID: \"026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"
Jul 30 00:54:24.368651 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.368544    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x89fq\" (UniqueName: \"kubernetes.io/projected/026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b-kube-api-access-x89fq\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9\" (UID: \"026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"
Jul 30 00:54:24.368651 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.368577    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9\" (UID: \"026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"
Jul 30 00:54:24.469671 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.469637    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9\" (UID: \"026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"
Jul 30 00:54:24.469671 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.469674    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9\" (UID: \"026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"
Jul 30 00:54:24.469888 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.469705    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x89fq\" (UniqueName: \"kubernetes.io/projected/026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b-kube-api-access-x89fq\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9\" (UID: \"026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"
Jul 30 00:54:24.469888 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.469733    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9\" (UID: \"026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"
Jul 30 00:54:24.469888 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.469781    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9\" (UID: \"026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"
Jul 30 00:54:24.469888 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.469820    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9\" (UID: \"026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"
Jul 30 00:54:24.470074 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.470049    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9\" (UID: \"026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"
Jul 30 00:54:24.470118 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.470078    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9\" (UID: \"026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"
Jul 30 00:54:24.470166 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.470144    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9\" (UID: \"026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"
Jul 30 00:54:24.472057 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.472032    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9\" (UID: \"026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"
Jul 30 00:54:24.472180 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.472165    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9\" (UID: \"026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"
Jul 30 00:54:24.478678 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.478660    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x89fq\" (UniqueName: \"kubernetes.io/projected/026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b-kube-api-access-x89fq\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9\" (UID: \"026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"
Jul 30 00:54:24.655323 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.655240    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"
Jul 30 00:54:24.796116 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.796087    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"]
Jul 30 00:54:24.797948 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:54:24.797914    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod026a16bb_b4ca_44b3_99b1_9ba46f2f9c6b.slice/crio-0bc38a27776923b01884fddfb9a6d7032aa50a0b786cd807471fd97147352b51 WatchSource:0}: Error finding container 0bc38a27776923b01884fddfb9a6d7032aa50a0b786cd807471fd97147352b51: Status 404 returned error can't find the container with id 0bc38a27776923b01884fddfb9a6d7032aa50a0b786cd807471fd97147352b51
Jul 30 00:54:24.799454 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:24.799428    2546 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 00:54:25.330083 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:25.330034    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9" event={"ID":"026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b","Type":"ContainerStarted","Data":"0bc38a27776923b01884fddfb9a6d7032aa50a0b786cd807471fd97147352b51"}
Jul 30 00:54:26.337858 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:26.337822    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9" event={"ID":"026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b","Type":"ContainerStarted","Data":"075af056584a11cfa4671e152b289c30f2908f936a003e7e17e5fa401c260886"}
Jul 30 00:54:26.357028 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:26.356966    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9" podStartSLOduration=1.244955734 podStartE2EDuration="2.356948766s" totalImagesPullingTime="1.111993032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:54:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:54:26.355047121 +0000 UTC m=+653.577780965" watchObservedRunningTime="2026-07-30 00:54:26.356948766 +0000 UTC m=+653.579682584"
Jul 30 00:54:34.655586 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:34.655514    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"
Jul 30 00:54:34.656017 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:34.655619    2546 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"
Jul 30 00:54:34.668069 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:34.668042    2546 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"
Jul 30 00:54:35.392865 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:54:35.392829    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9"
Jul 30 00:55:11.652964 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:11.652922    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"]
Jul 30 00:55:11.655503 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:11.655484    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"
Jul 30 00:55:11.657989 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:11.657969    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 30 00:55:11.666260 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:11.666240    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"]
Jul 30 00:55:11.675773 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:11.675753    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt\" (UID: \"b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"
Jul 30 00:55:11.675883 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:11.675785    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt\" (UID: \"b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"
Jul 30 00:55:11.675883 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:11.675819    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt\" (UID: \"b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"
Jul 30 00:55:11.676007 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:11.675894    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt\" (UID: \"b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"
Jul 30 00:55:11.676007 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:11.675929    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwhsh\" (UniqueName: \"kubernetes.io/projected/b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7-kube-api-access-cwhsh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt\" (UID: \"b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"
Jul 30 00:55:11.676007 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:11.675949    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt\" (UID: \"b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"
Jul 30 00:55:11.776768 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:11.776737    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt\" (UID: \"b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"
Jul 30 00:55:11.776768 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:11.776777    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwhsh\" (UniqueName: \"kubernetes.io/projected/b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7-kube-api-access-cwhsh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt\" (UID: \"b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"
Jul 30 00:55:11.776985 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:11.776797    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt\" (UID: \"b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"
Jul 30 00:55:11.776985 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:11.776850    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt\" (UID: \"b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"
Jul 30 00:55:11.776985 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:11.776871    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt\" (UID: \"b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"
Jul 30 00:55:11.776985 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:11.776900    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt\" (UID: \"b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"
Jul 30 00:55:11.777198 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:11.777147    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt\" (UID: \"b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"
Jul 30 00:55:11.777254 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:11.777223    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt\" (UID: \"b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"
Jul 30 00:55:11.777296 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:11.777251    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt\" (UID: \"b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"
Jul 30 00:55:11.779025 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:11.779004    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt\" (UID: \"b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"
Jul 30 00:55:11.779284 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:11.779267    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt\" (UID: \"b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"
Jul 30 00:55:11.784601 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:11.784582    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwhsh\" (UniqueName: \"kubernetes.io/projected/b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7-kube-api-access-cwhsh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt\" (UID: \"b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"
Jul 30 00:55:11.966511 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:11.966425    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"
Jul 30 00:55:12.102000 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:12.101973    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"]
Jul 30 00:55:12.104084 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:55:12.104057    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7cf65d6_77fa_4736_8b8e_b6fb863ed4d7.slice/crio-af100f046497f04fb394c4592ec1b0233e7eecdecd2d420f931fef713916e3a7 WatchSource:0}: Error finding container af100f046497f04fb394c4592ec1b0233e7eecdecd2d420f931fef713916e3a7: Status 404 returned error can't find the container with id af100f046497f04fb394c4592ec1b0233e7eecdecd2d420f931fef713916e3a7
Jul 30 00:55:12.563726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:12.563645    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt" event={"ID":"b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7","Type":"ContainerStarted","Data":"53d82d057c77ccfb2dee31c7d973439d307efeddad41973f0a04c5115426bc1c"}
Jul 30 00:55:12.563726 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:12.563695    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt" event={"ID":"b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7","Type":"ContainerStarted","Data":"af100f046497f04fb394c4592ec1b0233e7eecdecd2d420f931fef713916e3a7"}
Jul 30 00:55:12.581191 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:12.581143    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt" podStartSLOduration=1.389427771 podStartE2EDuration="1.581128364s" totalImagesPullingTime="191.700593ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:55:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:55:12.579474655 +0000 UTC m=+699.802208476" watchObservedRunningTime="2026-07-30 00:55:12.581128364 +0000 UTC m=+699.803862182"
Jul 30 00:55:21.966951 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:21.966911    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"
Jul 30 00:55:21.967384 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:21.966982    2546 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"
Jul 30 00:55:21.979377 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:21.979357    2546 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"
Jul 30 00:55:22.622531 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:55:22.622492    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt"
Jul 30 00:57:10.003633 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:57:10.003505    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-756cdf68f8-nv7bf"]
Jul 30 00:57:10.004188 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:57:10.003834    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-756cdf68f8-nv7bf" podUID="84fbcbec-79d9-42c4-a121-676adfab60e6" containerName="manager" containerID="cri-o://2cd60f96364126ffcbc937385278a4a01440cac81dadfce200f5058a87b776f7" gracePeriod=30
Jul 30 00:57:10.146302 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:57:10.146272    2546 generic.go:356] "Generic (PLEG): container finished" podID="84fbcbec-79d9-42c4-a121-676adfab60e6" containerID="2cd60f96364126ffcbc937385278a4a01440cac81dadfce200f5058a87b776f7" exitCode=0
Jul 30 00:57:10.146479 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:57:10.146349    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-756cdf68f8-nv7bf" event={"ID":"84fbcbec-79d9-42c4-a121-676adfab60e6","Type":"ContainerDied","Data":"2cd60f96364126ffcbc937385278a4a01440cac81dadfce200f5058a87b776f7"}
Jul 30 00:57:10.257501 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:57:10.257444    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-756cdf68f8-nv7bf"
Jul 30 00:57:10.324311 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:57:10.324275    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngvs6\" (UniqueName: \"kubernetes.io/projected/84fbcbec-79d9-42c4-a121-676adfab60e6-kube-api-access-ngvs6\") pod \"84fbcbec-79d9-42c4-a121-676adfab60e6\" (UID: \"84fbcbec-79d9-42c4-a121-676adfab60e6\") "
Jul 30 00:57:10.324497 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:57:10.324358    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84fbcbec-79d9-42c4-a121-676adfab60e6-webhook-cert\") pod \"84fbcbec-79d9-42c4-a121-676adfab60e6\" (UID: \"84fbcbec-79d9-42c4-a121-676adfab60e6\") "
Jul 30 00:57:10.326430 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:57:10.326395    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84fbcbec-79d9-42c4-a121-676adfab60e6-kube-api-access-ngvs6" (OuterVolumeSpecName: "kube-api-access-ngvs6") pod "84fbcbec-79d9-42c4-a121-676adfab60e6" (UID: "84fbcbec-79d9-42c4-a121-676adfab60e6"). InnerVolumeSpecName "kube-api-access-ngvs6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 00:57:10.326555 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:57:10.326477    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84fbcbec-79d9-42c4-a121-676adfab60e6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "84fbcbec-79d9-42c4-a121-676adfab60e6" (UID: "84fbcbec-79d9-42c4-a121-676adfab60e6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 00:57:10.425979 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:57:10.425948    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngvs6\" (UniqueName: \"kubernetes.io/projected/84fbcbec-79d9-42c4-a121-676adfab60e6-kube-api-access-ngvs6\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:57:10.425979 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:57:10.425986    2546 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84fbcbec-79d9-42c4-a121-676adfab60e6-webhook-cert\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:57:11.152794 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:57:11.152757    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-756cdf68f8-nv7bf" event={"ID":"84fbcbec-79d9-42c4-a121-676adfab60e6","Type":"ContainerDied","Data":"79810100cee86f2366f4c4d09d101872fc80ebe73ef6ab6e4240013be5248f71"}
Jul 30 00:57:11.152794 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:57:11.152788    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-756cdf68f8-nv7bf"
Jul 30 00:57:11.153296 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:57:11.152822    2546 scope.go:117] "RemoveContainer" containerID="2cd60f96364126ffcbc937385278a4a01440cac81dadfce200f5058a87b776f7"
Jul 30 00:57:11.173936 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:57:11.173906    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-756cdf68f8-nv7bf"]
Jul 30 00:57:11.179913 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:57:11.179887    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-756cdf68f8-nv7bf"]
Jul 30 00:57:11.358393 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:57:11.358353    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84fbcbec-79d9-42c4-a121-676adfab60e6" path="/var/lib/kubelet/pods/84fbcbec-79d9-42c4-a121-676adfab60e6/volumes"
Jul 30 00:58:11.952256 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:11.952218    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-756cdf68f8-fthmj"]
Jul 30 00:58:11.952769 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:11.952646    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84fbcbec-79d9-42c4-a121-676adfab60e6" containerName="manager"
Jul 30 00:58:11.952769 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:11.952659    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="84fbcbec-79d9-42c4-a121-676adfab60e6" containerName="manager"
Jul 30 00:58:11.952769 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:11.952720    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="84fbcbec-79d9-42c4-a121-676adfab60e6" containerName="manager"
Jul 30 00:58:11.955957 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:11.955936    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-756cdf68f8-fthmj"
Jul 30 00:58:11.958374 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:11.958354    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 30 00:58:11.958510 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:11.958354    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-q4wph\""
Jul 30 00:58:11.959446 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:11.959428    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 30 00:58:11.965947 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:11.965918    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-756cdf68f8-fthmj"]
Jul 30 00:58:12.063608 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:12.063562    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzxdl\" (UniqueName: \"kubernetes.io/projected/408eb6e9-fab9-43b4-b9ca-1019f7c575fe-kube-api-access-xzxdl\") pod \"maas-controller-756cdf68f8-fthmj\" (UID: \"408eb6e9-fab9-43b4-b9ca-1019f7c575fe\") " pod="opendatahub/maas-controller-756cdf68f8-fthmj"
Jul 30 00:58:12.063796 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:12.063673    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/408eb6e9-fab9-43b4-b9ca-1019f7c575fe-webhook-cert\") pod \"maas-controller-756cdf68f8-fthmj\" (UID: \"408eb6e9-fab9-43b4-b9ca-1019f7c575fe\") " pod="opendatahub/maas-controller-756cdf68f8-fthmj"
Jul 30 00:58:12.165033 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:12.164991    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzxdl\" (UniqueName: \"kubernetes.io/projected/408eb6e9-fab9-43b4-b9ca-1019f7c575fe-kube-api-access-xzxdl\") pod \"maas-controller-756cdf68f8-fthmj\" (UID: \"408eb6e9-fab9-43b4-b9ca-1019f7c575fe\") " pod="opendatahub/maas-controller-756cdf68f8-fthmj"
Jul 30 00:58:12.165207 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:12.165050    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/408eb6e9-fab9-43b4-b9ca-1019f7c575fe-webhook-cert\") pod \"maas-controller-756cdf68f8-fthmj\" (UID: \"408eb6e9-fab9-43b4-b9ca-1019f7c575fe\") " pod="opendatahub/maas-controller-756cdf68f8-fthmj"
Jul 30 00:58:12.167404 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:12.167375    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/408eb6e9-fab9-43b4-b9ca-1019f7c575fe-webhook-cert\") pod \"maas-controller-756cdf68f8-fthmj\" (UID: \"408eb6e9-fab9-43b4-b9ca-1019f7c575fe\") " pod="opendatahub/maas-controller-756cdf68f8-fthmj"
Jul 30 00:58:12.173379 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:12.173351    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzxdl\" (UniqueName: \"kubernetes.io/projected/408eb6e9-fab9-43b4-b9ca-1019f7c575fe-kube-api-access-xzxdl\") pod \"maas-controller-756cdf68f8-fthmj\" (UID: \"408eb6e9-fab9-43b4-b9ca-1019f7c575fe\") " pod="opendatahub/maas-controller-756cdf68f8-fthmj"
Jul 30 00:58:12.267924 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:12.267876    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-756cdf68f8-fthmj"
Jul 30 00:58:12.408400 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:12.408368    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-756cdf68f8-fthmj"]
Jul 30 00:58:12.410385 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:58:12.410348    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod408eb6e9_fab9_43b4_b9ca_1019f7c575fe.slice/crio-244924d2239f0cf92637a51da5649cc6e68f6a1c086df6dd6e413b4302a7ccb6 WatchSource:0}: Error finding container 244924d2239f0cf92637a51da5649cc6e68f6a1c086df6dd6e413b4302a7ccb6: Status 404 returned error can't find the container with id 244924d2239f0cf92637a51da5649cc6e68f6a1c086df6dd6e413b4302a7ccb6
Jul 30 00:58:12.459979 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:12.459944    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-756cdf68f8-fthmj" event={"ID":"408eb6e9-fab9-43b4-b9ca-1019f7c575fe","Type":"ContainerStarted","Data":"244924d2239f0cf92637a51da5649cc6e68f6a1c086df6dd6e413b4302a7ccb6"}
Jul 30 00:58:13.467314 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:13.467273    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-756cdf68f8-fthmj" event={"ID":"408eb6e9-fab9-43b4-b9ca-1019f7c575fe","Type":"ContainerStarted","Data":"fef916721e8ad0e4471ba7d5009df2a45d144e646ba7530557dc972ac657c06e"}
Jul 30 00:58:13.467788 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:13.467338    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-756cdf68f8-fthmj"
Jul 30 00:58:13.484246 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:13.484192    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-756cdf68f8-fthmj" podStartSLOduration=1.544092762 podStartE2EDuration="2.48417695s" totalImagesPullingTime="940.084188ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:58:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:58:13.482883912 +0000 UTC m=+880.705617731" watchObservedRunningTime="2026-07-30 00:58:13.48417695 +0000 UTC m=+880.706910766"
Jul 30 00:58:24.474781 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:24.474747    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-756cdf68f8-fthmj"
Jul 30 00:58:29.120214 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:29.120173    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-756cdf68f8-fthmj"]
Jul 30 00:58:29.120696 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:29.120412    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-756cdf68f8-fthmj" podUID="408eb6e9-fab9-43b4-b9ca-1019f7c575fe" containerName="manager" containerID="cri-o://fef916721e8ad0e4471ba7d5009df2a45d144e646ba7530557dc972ac657c06e" gracePeriod=30
Jul 30 00:58:29.377149 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:29.377086    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-756cdf68f8-fthmj"
Jul 30 00:58:29.528471 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:29.528425    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/408eb6e9-fab9-43b4-b9ca-1019f7c575fe-webhook-cert\") pod \"408eb6e9-fab9-43b4-b9ca-1019f7c575fe\" (UID: \"408eb6e9-fab9-43b4-b9ca-1019f7c575fe\") "
Jul 30 00:58:29.528701 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:29.528483    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzxdl\" (UniqueName: \"kubernetes.io/projected/408eb6e9-fab9-43b4-b9ca-1019f7c575fe-kube-api-access-xzxdl\") pod \"408eb6e9-fab9-43b4-b9ca-1019f7c575fe\" (UID: \"408eb6e9-fab9-43b4-b9ca-1019f7c575fe\") "
Jul 30 00:58:29.530559 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:29.530534    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/408eb6e9-fab9-43b4-b9ca-1019f7c575fe-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "408eb6e9-fab9-43b4-b9ca-1019f7c575fe" (UID: "408eb6e9-fab9-43b4-b9ca-1019f7c575fe"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 00:58:29.530719 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:29.530703    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/408eb6e9-fab9-43b4-b9ca-1019f7c575fe-kube-api-access-xzxdl" (OuterVolumeSpecName: "kube-api-access-xzxdl") pod "408eb6e9-fab9-43b4-b9ca-1019f7c575fe" (UID: "408eb6e9-fab9-43b4-b9ca-1019f7c575fe"). InnerVolumeSpecName "kube-api-access-xzxdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 00:58:29.550926 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:29.550892    2546 generic.go:356] "Generic (PLEG): container finished" podID="408eb6e9-fab9-43b4-b9ca-1019f7c575fe" containerID="fef916721e8ad0e4471ba7d5009df2a45d144e646ba7530557dc972ac657c06e" exitCode=0
Jul 30 00:58:29.551074 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:29.550939    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-756cdf68f8-fthmj"
Jul 30 00:58:29.551074 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:29.550943    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-756cdf68f8-fthmj" event={"ID":"408eb6e9-fab9-43b4-b9ca-1019f7c575fe","Type":"ContainerDied","Data":"fef916721e8ad0e4471ba7d5009df2a45d144e646ba7530557dc972ac657c06e"}
Jul 30 00:58:29.551074 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:29.550996    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-756cdf68f8-fthmj" event={"ID":"408eb6e9-fab9-43b4-b9ca-1019f7c575fe","Type":"ContainerDied","Data":"244924d2239f0cf92637a51da5649cc6e68f6a1c086df6dd6e413b4302a7ccb6"}
Jul 30 00:58:29.551074 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:29.551026    2546 scope.go:117] "RemoveContainer" containerID="fef916721e8ad0e4471ba7d5009df2a45d144e646ba7530557dc972ac657c06e"
Jul 30 00:58:29.562741 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:29.562717    2546 scope.go:117] "RemoveContainer" containerID="fef916721e8ad0e4471ba7d5009df2a45d144e646ba7530557dc972ac657c06e"
Jul 30 00:58:29.563035 ip-10-0-141-85 kubenswrapper[2546]: E0730 00:58:29.563016    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fef916721e8ad0e4471ba7d5009df2a45d144e646ba7530557dc972ac657c06e\": container with ID starting with fef916721e8ad0e4471ba7d5009df2a45d144e646ba7530557dc972ac657c06e not found: ID does not exist" containerID="fef916721e8ad0e4471ba7d5009df2a45d144e646ba7530557dc972ac657c06e"
Jul 30 00:58:29.563087 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:29.563043    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fef916721e8ad0e4471ba7d5009df2a45d144e646ba7530557dc972ac657c06e"} err="failed to get container status \"fef916721e8ad0e4471ba7d5009df2a45d144e646ba7530557dc972ac657c06e\": rpc error: code = NotFound desc = could not find container \"fef916721e8ad0e4471ba7d5009df2a45d144e646ba7530557dc972ac657c06e\": container with ID starting with fef916721e8ad0e4471ba7d5009df2a45d144e646ba7530557dc972ac657c06e not found: ID does not exist"
Jul 30 00:58:29.574578 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:29.574547    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-756cdf68f8-fthmj"]
Jul 30 00:58:29.578374 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:29.578350    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-756cdf68f8-fthmj"]
Jul 30 00:58:29.629927 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:29.629843    2546 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/408eb6e9-fab9-43b4-b9ca-1019f7c575fe-webhook-cert\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:58:29.629927 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:29.629872    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzxdl\" (UniqueName: \"kubernetes.io/projected/408eb6e9-fab9-43b4-b9ca-1019f7c575fe-kube-api-access-xzxdl\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 00:58:30.378374 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:30.378328    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-756cdf68f8-smlzn"]
Jul 30 00:58:30.378960 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:30.378935    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="408eb6e9-fab9-43b4-b9ca-1019f7c575fe" containerName="manager"
Jul 30 00:58:30.378960 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:30.378957    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="408eb6e9-fab9-43b4-b9ca-1019f7c575fe" containerName="manager"
Jul 30 00:58:30.379103 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:30.379087    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="408eb6e9-fab9-43b4-b9ca-1019f7c575fe" containerName="manager"
Jul 30 00:58:30.383474 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:30.383450    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-756cdf68f8-smlzn"
Jul 30 00:58:30.386883 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:30.386863    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-q4wph\""
Jul 30 00:58:30.387031 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:30.386864    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 30 00:58:30.387031 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:30.386920    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 30 00:58:30.391949 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:30.391927    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-756cdf68f8-smlzn"]
Jul 30 00:58:30.539133 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:30.539083    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2bcc1a1a-24a5-4fa3-8d47-c2a498047a18-webhook-cert\") pod \"maas-controller-756cdf68f8-smlzn\" (UID: \"2bcc1a1a-24a5-4fa3-8d47-c2a498047a18\") " pod="opendatahub/maas-controller-756cdf68f8-smlzn"
Jul 30 00:58:30.539324 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:30.539170    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cvn2\" (UniqueName: \"kubernetes.io/projected/2bcc1a1a-24a5-4fa3-8d47-c2a498047a18-kube-api-access-8cvn2\") pod \"maas-controller-756cdf68f8-smlzn\" (UID: \"2bcc1a1a-24a5-4fa3-8d47-c2a498047a18\") " pod="opendatahub/maas-controller-756cdf68f8-smlzn"
Jul 30 00:58:30.639665 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:30.639580    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2bcc1a1a-24a5-4fa3-8d47-c2a498047a18-webhook-cert\") pod \"maas-controller-756cdf68f8-smlzn\" (UID: \"2bcc1a1a-24a5-4fa3-8d47-c2a498047a18\") " pod="opendatahub/maas-controller-756cdf68f8-smlzn"
Jul 30 00:58:30.639665 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:30.639647    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cvn2\" (UniqueName: \"kubernetes.io/projected/2bcc1a1a-24a5-4fa3-8d47-c2a498047a18-kube-api-access-8cvn2\") pod \"maas-controller-756cdf68f8-smlzn\" (UID: \"2bcc1a1a-24a5-4fa3-8d47-c2a498047a18\") " pod="opendatahub/maas-controller-756cdf68f8-smlzn"
Jul 30 00:58:30.642125 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:30.642106    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2bcc1a1a-24a5-4fa3-8d47-c2a498047a18-webhook-cert\") pod \"maas-controller-756cdf68f8-smlzn\" (UID: \"2bcc1a1a-24a5-4fa3-8d47-c2a498047a18\") " pod="opendatahub/maas-controller-756cdf68f8-smlzn"
Jul 30 00:58:30.648427 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:30.648397    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cvn2\" (UniqueName: \"kubernetes.io/projected/2bcc1a1a-24a5-4fa3-8d47-c2a498047a18-kube-api-access-8cvn2\") pod \"maas-controller-756cdf68f8-smlzn\" (UID: \"2bcc1a1a-24a5-4fa3-8d47-c2a498047a18\") " pod="opendatahub/maas-controller-756cdf68f8-smlzn"
Jul 30 00:58:30.696742 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:30.696699    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-756cdf68f8-smlzn"
Jul 30 00:58:30.838884 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:30.838854    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-756cdf68f8-smlzn"]
Jul 30 00:58:30.841186 ip-10-0-141-85 kubenswrapper[2546]: W0730 00:58:30.841155    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bcc1a1a_24a5_4fa3_8d47_c2a498047a18.slice/crio-56d7de644586886e6f5d908ad3bb1b71b8f2752afebd306fbd5a4edb40de71a0 WatchSource:0}: Error finding container 56d7de644586886e6f5d908ad3bb1b71b8f2752afebd306fbd5a4edb40de71a0: Status 404 returned error can't find the container with id 56d7de644586886e6f5d908ad3bb1b71b8f2752afebd306fbd5a4edb40de71a0
Jul 30 00:58:31.359088 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:31.359053    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="408eb6e9-fab9-43b4-b9ca-1019f7c575fe" path="/var/lib/kubelet/pods/408eb6e9-fab9-43b4-b9ca-1019f7c575fe/volumes"
Jul 30 00:58:31.563757 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:31.563717    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-756cdf68f8-smlzn" event={"ID":"2bcc1a1a-24a5-4fa3-8d47-c2a498047a18","Type":"ContainerStarted","Data":"78347905bd7aed19f1d74c95441791e91182b9fe185c9840a23137c0bc1cf21e"}
Jul 30 00:58:31.564175 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:31.563782    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-756cdf68f8-smlzn"
Jul 30 00:58:31.564175 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:31.563805    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-756cdf68f8-smlzn" event={"ID":"2bcc1a1a-24a5-4fa3-8d47-c2a498047a18","Type":"ContainerStarted","Data":"56d7de644586886e6f5d908ad3bb1b71b8f2752afebd306fbd5a4edb40de71a0"}
Jul 30 00:58:31.583804 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:31.583747    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-756cdf68f8-smlzn" podStartSLOduration=1.231469291 podStartE2EDuration="1.583730075s" totalImagesPullingTime="352.260784ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 00:58:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 00:58:31.581011564 +0000 UTC m=+898.803745382" watchObservedRunningTime="2026-07-30 00:58:31.583730075 +0000 UTC m=+898.806463893"
Jul 30 00:58:33.220306 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:33.220275    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 00:58:33.221443 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:33.221419    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 00:58:42.570842 ip-10-0-141-85 kubenswrapper[2546]: I0730 00:58:42.570808    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-756cdf68f8-smlzn"
Jul 30 01:03:33.248770 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:03:33.248680    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 01:03:33.252001 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:03:33.251974    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 01:05:41.164654 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:05:41.164613    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t9jvr"]
Jul 30 01:05:41.167141 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:05:41.164871    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t9jvr" podUID="d0ad9948-0052-4a70-854c-58ee88e5e545" containerName="manager" containerID="cri-o://d1264833516c3341af9bfadf6d98e33c1fba74375f4b78a636a7986cfced726e" gracePeriod=10
Jul 30 01:05:41.431718 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:05:41.431692    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t9jvr"
Jul 30 01:05:41.542753 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:05:41.542696    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tggrr\" (UniqueName: \"kubernetes.io/projected/d0ad9948-0052-4a70-854c-58ee88e5e545-kube-api-access-tggrr\") pod \"d0ad9948-0052-4a70-854c-58ee88e5e545\" (UID: \"d0ad9948-0052-4a70-854c-58ee88e5e545\") "
Jul 30 01:05:41.542962 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:05:41.542811    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0ad9948-0052-4a70-854c-58ee88e5e545-extensions-socket-volume\") pod \"d0ad9948-0052-4a70-854c-58ee88e5e545\" (UID: \"d0ad9948-0052-4a70-854c-58ee88e5e545\") "
Jul 30 01:05:41.543192 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:05:41.543166    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ad9948-0052-4a70-854c-58ee88e5e545-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d0ad9948-0052-4a70-854c-58ee88e5e545" (UID: "d0ad9948-0052-4a70-854c-58ee88e5e545"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:05:41.545040 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:05:41.545012    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0ad9948-0052-4a70-854c-58ee88e5e545-kube-api-access-tggrr" (OuterVolumeSpecName: "kube-api-access-tggrr") pod "d0ad9948-0052-4a70-854c-58ee88e5e545" (UID: "d0ad9948-0052-4a70-854c-58ee88e5e545"). InnerVolumeSpecName "kube-api-access-tggrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:05:41.644159 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:05:41.644099    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tggrr\" (UniqueName: \"kubernetes.io/projected/d0ad9948-0052-4a70-854c-58ee88e5e545-kube-api-access-tggrr\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:05:41.644159 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:05:41.644150    2546 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0ad9948-0052-4a70-854c-58ee88e5e545-extensions-socket-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:05:41.728888 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:05:41.728851    2546 generic.go:356] "Generic (PLEG): container finished" podID="d0ad9948-0052-4a70-854c-58ee88e5e545" containerID="d1264833516c3341af9bfadf6d98e33c1fba74375f4b78a636a7986cfced726e" exitCode=0
Jul 30 01:05:41.729101 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:05:41.728900    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t9jvr"
Jul 30 01:05:41.729101 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:05:41.728932    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t9jvr" event={"ID":"d0ad9948-0052-4a70-854c-58ee88e5e545","Type":"ContainerDied","Data":"d1264833516c3341af9bfadf6d98e33c1fba74375f4b78a636a7986cfced726e"}
Jul 30 01:05:41.729101 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:05:41.728987    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t9jvr" event={"ID":"d0ad9948-0052-4a70-854c-58ee88e5e545","Type":"ContainerDied","Data":"237ce58c316e74e988bf96bcaf89c413b40a6dfe0d889464672205e7a8c74aff"}
Jul 30 01:05:41.729101 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:05:41.729009    2546 scope.go:117] "RemoveContainer" containerID="d1264833516c3341af9bfadf6d98e33c1fba74375f4b78a636a7986cfced726e"
Jul 30 01:05:41.738953 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:05:41.738933    2546 scope.go:117] "RemoveContainer" containerID="d1264833516c3341af9bfadf6d98e33c1fba74375f4b78a636a7986cfced726e"
Jul 30 01:05:41.739264 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:05:41.739240    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1264833516c3341af9bfadf6d98e33c1fba74375f4b78a636a7986cfced726e\": container with ID starting with d1264833516c3341af9bfadf6d98e33c1fba74375f4b78a636a7986cfced726e not found: ID does not exist" containerID="d1264833516c3341af9bfadf6d98e33c1fba74375f4b78a636a7986cfced726e"
Jul 30 01:05:41.739371 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:05:41.739274    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1264833516c3341af9bfadf6d98e33c1fba74375f4b78a636a7986cfced726e"} err="failed to get container status \"d1264833516c3341af9bfadf6d98e33c1fba74375f4b78a636a7986cfced726e\": rpc error: code = NotFound desc = could not find container \"d1264833516c3341af9bfadf6d98e33c1fba74375f4b78a636a7986cfced726e\": container with ID starting with d1264833516c3341af9bfadf6d98e33c1fba74375f4b78a636a7986cfced726e not found: ID does not exist"
Jul 30 01:05:41.752028 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:05:41.751997    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t9jvr"]
Jul 30 01:05:41.756151 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:05:41.756121    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t9jvr"]
Jul 30 01:05:43.358242 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:05:43.358207    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0ad9948-0052-4a70-854c-58ee88e5e545" path="/var/lib/kubelet/pods/d0ad9948-0052-4a70-854c-58ee88e5e545/volumes"
Jul 30 01:06:41.856329 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:06:41.856293    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ftprg"]
Jul 30 01:06:41.856775 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:06:41.856728    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0ad9948-0052-4a70-854c-58ee88e5e545" containerName="manager"
Jul 30 01:06:41.856775 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:06:41.856741    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0ad9948-0052-4a70-854c-58ee88e5e545" containerName="manager"
Jul 30 01:06:41.856849 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:06:41.856803    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0ad9948-0052-4a70-854c-58ee88e5e545" containerName="manager"
Jul 30 01:06:41.861047 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:06:41.861028    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ftprg"
Jul 30 01:06:41.864377 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:06:41.864356    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dgwdc\""
Jul 30 01:06:41.870593 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:06:41.870564    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ftprg"]
Jul 30 01:06:41.872559 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:06:41.872516    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9x4n\" (UniqueName: \"kubernetes.io/projected/f39e270c-2047-478a-89bf-86e7b630271f-kube-api-access-x9x4n\") pod \"kuadrant-operator-controller-manager-86878c769f-ftprg\" (UID: \"f39e270c-2047-478a-89bf-86e7b630271f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ftprg"
Jul 30 01:06:41.873047 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:06:41.873025    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f39e270c-2047-478a-89bf-86e7b630271f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ftprg\" (UID: \"f39e270c-2047-478a-89bf-86e7b630271f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ftprg"
Jul 30 01:06:41.973960 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:06:41.973920    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9x4n\" (UniqueName: \"kubernetes.io/projected/f39e270c-2047-478a-89bf-86e7b630271f-kube-api-access-x9x4n\") pod \"kuadrant-operator-controller-manager-86878c769f-ftprg\" (UID: \"f39e270c-2047-478a-89bf-86e7b630271f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ftprg"
Jul 30 01:06:41.974135 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:06:41.973970    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f39e270c-2047-478a-89bf-86e7b630271f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ftprg\" (UID: \"f39e270c-2047-478a-89bf-86e7b630271f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ftprg"
Jul 30 01:06:41.974346 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:06:41.974329    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f39e270c-2047-478a-89bf-86e7b630271f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ftprg\" (UID: \"f39e270c-2047-478a-89bf-86e7b630271f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ftprg"
Jul 30 01:06:41.985965 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:06:41.985933    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9x4n\" (UniqueName: \"kubernetes.io/projected/f39e270c-2047-478a-89bf-86e7b630271f-kube-api-access-x9x4n\") pod \"kuadrant-operator-controller-manager-86878c769f-ftprg\" (UID: \"f39e270c-2047-478a-89bf-86e7b630271f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ftprg"
Jul 30 01:06:42.173201 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:06:42.173105    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ftprg"
Jul 30 01:06:42.323096 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:06:42.323071    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ftprg"]
Jul 30 01:06:42.325278 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:06:42.325239    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf39e270c_2047_478a_89bf_86e7b630271f.slice/crio-3c8fb8220a2b21c24e3aba190c766ce86e8da9102bac4c5d710b4666aacb1740 WatchSource:0}: Error finding container 3c8fb8220a2b21c24e3aba190c766ce86e8da9102bac4c5d710b4666aacb1740: Status 404 returned error can't find the container with id 3c8fb8220a2b21c24e3aba190c766ce86e8da9102bac4c5d710b4666aacb1740
Jul 30 01:06:43.030598 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:06:43.030563    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ftprg" event={"ID":"f39e270c-2047-478a-89bf-86e7b630271f","Type":"ContainerStarted","Data":"8a41350cac34babc224e94e37d7e671b63c187f56bc5fd00d00efe77852ece17"}
Jul 30 01:06:43.031025 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:06:43.030621    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ftprg" event={"ID":"f39e270c-2047-478a-89bf-86e7b630271f","Type":"ContainerStarted","Data":"3c8fb8220a2b21c24e3aba190c766ce86e8da9102bac4c5d710b4666aacb1740"}
Jul 30 01:06:43.031025 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:06:43.030646    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ftprg"
Jul 30 01:06:43.051752 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:06:43.051691    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ftprg" podStartSLOduration=2.051674107 podStartE2EDuration="2.051674107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:06:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:06:43.049258806 +0000 UTC m=+1390.271992625" watchObservedRunningTime="2026-07-30 01:06:43.051674107 +0000 UTC m=+1390.274407925"
Jul 30 01:06:54.038457 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:06:54.038423    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ftprg"
Jul 30 01:08:33.281690 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:08:33.281658    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 01:08:33.282370 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:08:33.282350    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 01:13:33.321389 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:13:33.321363    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 01:13:33.322009 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:13:33.321981    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 01:15:37.180551 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:37.180498    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp"]
Jul 30 01:15:37.184020 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:37.183998    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp"
Jul 30 01:15:37.186781 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:37.186760    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-76aaf06c-serving-cert\""
Jul 30 01:15:37.196060 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:37.196038    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp"]
Jul 30 01:15:37.227767 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:37.227739    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da0227be-d5f6-46e5-8ccd-dbbd72632683-maas-api-tls\") pod \"maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp\" (UID: \"da0227be-d5f6-46e5-8ccd-dbbd72632683\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp"
Jul 30 01:15:37.227925 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:37.227826    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da0227be-d5f6-46e5-8ccd-dbbd72632683-openshift-service-ca\") pod \"maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp\" (UID: \"da0227be-d5f6-46e5-8ccd-dbbd72632683\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp"
Jul 30 01:15:37.227925 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:37.227848    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sscmd\" (UniqueName: \"kubernetes.io/projected/da0227be-d5f6-46e5-8ccd-dbbd72632683-kube-api-access-sscmd\") pod \"maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp\" (UID: \"da0227be-d5f6-46e5-8ccd-dbbd72632683\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp"
Jul 30 01:15:37.328491 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:37.328455    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da0227be-d5f6-46e5-8ccd-dbbd72632683-openshift-service-ca\") pod \"maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp\" (UID: \"da0227be-d5f6-46e5-8ccd-dbbd72632683\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp"
Jul 30 01:15:37.328491 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:37.328500    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sscmd\" (UniqueName: \"kubernetes.io/projected/da0227be-d5f6-46e5-8ccd-dbbd72632683-kube-api-access-sscmd\") pod \"maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp\" (UID: \"da0227be-d5f6-46e5-8ccd-dbbd72632683\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp"
Jul 30 01:15:37.328749 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:37.328701    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da0227be-d5f6-46e5-8ccd-dbbd72632683-maas-api-tls\") pod \"maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp\" (UID: \"da0227be-d5f6-46e5-8ccd-dbbd72632683\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp"
Jul 30 01:15:37.328845 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:15:37.328830    2546 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-serving-cert: secret "maas-api-e2e-ait-76aaf06c-serving-cert" not found
Jul 30 01:15:37.328910 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:15:37.328900    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da0227be-d5f6-46e5-8ccd-dbbd72632683-maas-api-tls podName:da0227be-d5f6-46e5-8ccd-dbbd72632683 nodeName:}" failed. No retries permitted until 2026-07-30 01:15:37.828883346 +0000 UTC m=+1925.051617142 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/da0227be-d5f6-46e5-8ccd-dbbd72632683-maas-api-tls") pod "maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp" (UID: "da0227be-d5f6-46e5-8ccd-dbbd72632683") : secret "maas-api-e2e-ait-76aaf06c-serving-cert" not found
Jul 30 01:15:37.329181 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:37.329158    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da0227be-d5f6-46e5-8ccd-dbbd72632683-openshift-service-ca\") pod \"maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp\" (UID: \"da0227be-d5f6-46e5-8ccd-dbbd72632683\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp"
Jul 30 01:15:37.338774 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:37.338750    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sscmd\" (UniqueName: \"kubernetes.io/projected/da0227be-d5f6-46e5-8ccd-dbbd72632683-kube-api-access-sscmd\") pod \"maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp\" (UID: \"da0227be-d5f6-46e5-8ccd-dbbd72632683\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp"
Jul 30 01:15:37.833139 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:37.833099    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da0227be-d5f6-46e5-8ccd-dbbd72632683-maas-api-tls\") pod \"maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp\" (UID: \"da0227be-d5f6-46e5-8ccd-dbbd72632683\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp"
Jul 30 01:15:37.835641 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:37.835618    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da0227be-d5f6-46e5-8ccd-dbbd72632683-maas-api-tls\") pod \"maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp\" (UID: \"da0227be-d5f6-46e5-8ccd-dbbd72632683\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp"
Jul 30 01:15:37.940290 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:37.940250    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp"]
Jul 30 01:15:37.944020 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:37.943998    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp"
Jul 30 01:15:37.946735 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:37.946716    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-76aaf06c\""
Jul 30 01:15:37.955571 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:37.955545    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp"]
Jul 30 01:15:37.967399 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:37.967369    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9"]
Jul 30 01:15:37.970925 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:37.970900    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9"
Jul 30 01:15:37.973989 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:37.973966    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-76aaf06c-dockercfg-r9x2n\""
Jul 30 01:15:37.982195 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:37.982164    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9"]
Jul 30 01:15:38.034854 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.034819    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zd6n\" (UniqueName: \"kubernetes.io/projected/ee5f4e73-b40a-4d51-80ed-826c65767385-kube-api-access-2zd6n\") pod \"payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp\" (UID: \"ee5f4e73-b40a-4d51-80ed-826c65767385\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp"
Jul 30 01:15:38.034854 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.034855    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8c75281-d1d4-45f2-b625-774a67216bb4-plugins-config-volume\") pod \"payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9\" (UID: \"b8c75281-d1d4-45f2-b625-774a67216bb4\") " pod="openshift-ingress/payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9"
Jul 30 01:15:38.035052 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.034874    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l89br\" (UniqueName: \"kubernetes.io/projected/b8c75281-d1d4-45f2-b625-774a67216bb4-kube-api-access-l89br\") pod \"payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9\" (UID: \"b8c75281-d1d4-45f2-b625-774a67216bb4\") " pod="openshift-ingress/payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9"
Jul 30 01:15:38.035052 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.034891    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee5f4e73-b40a-4d51-80ed-826c65767385-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp\" (UID: \"ee5f4e73-b40a-4d51-80ed-826c65767385\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp"
Jul 30 01:15:38.095848 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.095757    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp"
Jul 30 01:15:38.136369 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.136322    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zd6n\" (UniqueName: \"kubernetes.io/projected/ee5f4e73-b40a-4d51-80ed-826c65767385-kube-api-access-2zd6n\") pod \"payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp\" (UID: \"ee5f4e73-b40a-4d51-80ed-826c65767385\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp"
Jul 30 01:15:38.136553 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.136378    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8c75281-d1d4-45f2-b625-774a67216bb4-plugins-config-volume\") pod \"payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9\" (UID: \"b8c75281-d1d4-45f2-b625-774a67216bb4\") " pod="openshift-ingress/payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9"
Jul 30 01:15:38.136553 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.136417    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l89br\" (UniqueName: \"kubernetes.io/projected/b8c75281-d1d4-45f2-b625-774a67216bb4-kube-api-access-l89br\") pod \"payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9\" (UID: \"b8c75281-d1d4-45f2-b625-774a67216bb4\") " pod="openshift-ingress/payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9"
Jul 30 01:15:38.136553 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.136448    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee5f4e73-b40a-4d51-80ed-826c65767385-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp\" (UID: \"ee5f4e73-b40a-4d51-80ed-826c65767385\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp"
Jul 30 01:15:38.137159 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.137131    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8c75281-d1d4-45f2-b625-774a67216bb4-plugins-config-volume\") pod \"payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9\" (UID: \"b8c75281-d1d4-45f2-b625-774a67216bb4\") " pod="openshift-ingress/payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9"
Jul 30 01:15:38.137286 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.137210    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee5f4e73-b40a-4d51-80ed-826c65767385-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp\" (UID: \"ee5f4e73-b40a-4d51-80ed-826c65767385\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp"
Jul 30 01:15:38.146145 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.146118    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l89br\" (UniqueName: \"kubernetes.io/projected/b8c75281-d1d4-45f2-b625-774a67216bb4-kube-api-access-l89br\") pod \"payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9\" (UID: \"b8c75281-d1d4-45f2-b625-774a67216bb4\") " pod="openshift-ingress/payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9"
Jul 30 01:15:38.146323 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.146300    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zd6n\" (UniqueName: \"kubernetes.io/projected/ee5f4e73-b40a-4d51-80ed-826c65767385-kube-api-access-2zd6n\") pod \"payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp\" (UID: \"ee5f4e73-b40a-4d51-80ed-826c65767385\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp"
Jul 30 01:15:38.247791 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.247765    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp"]
Jul 30 01:15:38.250049 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:15:38.250021    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda0227be_d5f6_46e5_8ccd_dbbd72632683.slice/crio-fddb4df3be6255f783d90d097a932bc49307e2c539bfef73621be730119297a7 WatchSource:0}: Error finding container fddb4df3be6255f783d90d097a932bc49307e2c539bfef73621be730119297a7: Status 404 returned error can't find the container with id fddb4df3be6255f783d90d097a932bc49307e2c539bfef73621be730119297a7
Jul 30 01:15:38.251342 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.251325    2546 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 01:15:38.256085 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.255969    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp"
Jul 30 01:15:38.283126 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.283099    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9"
Jul 30 01:15:38.417147 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.416765    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp"]
Jul 30 01:15:38.417276 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:15:38.417249    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee5f4e73_b40a_4d51_80ed_826c65767385.slice/crio-4de941133a8f334836bdd809589b96dd2f6a768b615e48c42f3ad6e4d724e784 WatchSource:0}: Error finding container 4de941133a8f334836bdd809589b96dd2f6a768b615e48c42f3ad6e4d724e784: Status 404 returned error can't find the container with id 4de941133a8f334836bdd809589b96dd2f6a768b615e48c42f3ad6e4d724e784
Jul 30 01:15:38.454029 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.453977    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9"]
Jul 30 01:15:38.462139 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:15:38.462101    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8c75281_d1d4_45f2_b625_774a67216bb4.slice/crio-b49b8c2d69cd74f3f425b5eb01c00728c552dd294f90d7b8f09a5259ef48d5c8 WatchSource:0}: Error finding container b49b8c2d69cd74f3f425b5eb01c00728c552dd294f90d7b8f09a5259ef48d5c8: Status 404 returned error can't find the container with id b49b8c2d69cd74f3f425b5eb01c00728c552dd294f90d7b8f09a5259ef48d5c8
Jul 30 01:15:38.766049 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.766010    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9" event={"ID":"b8c75281-d1d4-45f2-b625-774a67216bb4","Type":"ContainerStarted","Data":"c5cf606256ee52c4eb331f83953cd9e7eb815361ff78f01c3b40a5b62276c64a"}
Jul 30 01:15:38.766215 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.766084    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9" event={"ID":"b8c75281-d1d4-45f2-b625-774a67216bb4","Type":"ContainerStarted","Data":"b49b8c2d69cd74f3f425b5eb01c00728c552dd294f90d7b8f09a5259ef48d5c8"}
Jul 30 01:15:38.766215 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.766208    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9"
Jul 30 01:15:38.767331 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.767305    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp" event={"ID":"da0227be-d5f6-46e5-8ccd-dbbd72632683","Type":"ContainerStarted","Data":"fddb4df3be6255f783d90d097a932bc49307e2c539bfef73621be730119297a7"}
Jul 30 01:15:38.768775 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.768753    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp" event={"ID":"ee5f4e73-b40a-4d51-80ed-826c65767385","Type":"ContainerStarted","Data":"c2879a93ce2773b2314ba26511c1dea2c82a60463dcf345e8fb0046a58d37acf"}
Jul 30 01:15:38.768908 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.768789    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp" event={"ID":"ee5f4e73-b40a-4d51-80ed-826c65767385","Type":"ContainerStarted","Data":"4de941133a8f334836bdd809589b96dd2f6a768b615e48c42f3ad6e4d724e784"}
Jul 30 01:15:38.768908 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.768900    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp"
Jul 30 01:15:38.810614 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.810556    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp" podStartSLOduration=1.810534304 podStartE2EDuration="1.810534304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:15:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:15:38.80910472 +0000 UTC m=+1926.031838539" watchObservedRunningTime="2026-07-30 01:15:38.810534304 +0000 UTC m=+1926.033268119"
Jul 30 01:15:38.811880 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:38.811841    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9" podStartSLOduration=1.811827914 podStartE2EDuration="1.811827914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:15:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:15:38.788840574 +0000 UTC m=+1926.011574393" watchObservedRunningTime="2026-07-30 01:15:38.811827914 +0000 UTC m=+1926.034561734"
Jul 30 01:15:39.775515 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:39.775471    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp" event={"ID":"da0227be-d5f6-46e5-8ccd-dbbd72632683","Type":"ContainerStarted","Data":"6bf8c7df9a2ff0651e69ab75e665236d11cc86515a92cd3d7f9a60f290cee66b"}
Jul 30 01:15:39.776016 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:39.775835    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp"
Jul 30 01:15:39.797736 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:39.797668    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp" podStartSLOduration=2.331804131 podStartE2EDuration="2.797647564s" totalImagesPullingTime="465.843433ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:15:39.795105405 +0000 UTC m=+1927.017839225" watchObservedRunningTime="2026-07-30 01:15:39.797647564 +0000 UTC m=+1927.020381384"
Jul 30 01:15:45.788266 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:45.788235    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp"
Jul 30 01:15:49.777732 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:49.777699    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9"
Jul 30 01:15:49.778135 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:15:49.777821    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp"
Jul 30 01:16:07.665615 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:07.665578    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp"]
Jul 30 01:16:07.666092 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:07.665827    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp" podUID="da0227be-d5f6-46e5-8ccd-dbbd72632683" containerName="maas-api" containerID="cri-o://6bf8c7df9a2ff0651e69ab75e665236d11cc86515a92cd3d7f9a60f290cee66b" gracePeriod=30
Jul 30 01:16:07.729475 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:07.729436    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9"]
Jul 30 01:16:07.729769 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:07.729738    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9" podUID="b8c75281-d1d4-45f2-b625-774a67216bb4" containerName="payload-processing" containerID="cri-o://c5cf606256ee52c4eb331f83953cd9e7eb815361ff78f01c3b40a5b62276c64a" gracePeriod=30
Jul 30 01:16:07.737345 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:07.737317    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp"]
Jul 30 01:16:07.737758 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:07.737721    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp" podUID="ee5f4e73-b40a-4d51-80ed-826c65767385" containerName="payload-pre-processing" containerID="cri-o://c2879a93ce2773b2314ba26511c1dea2c82a60463dcf345e8fb0046a58d37acf" gracePeriod=30
Jul 30 01:16:07.925673 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:07.925574    2546 generic.go:356] "Generic (PLEG): container finished" podID="b8c75281-d1d4-45f2-b625-774a67216bb4" containerID="c5cf606256ee52c4eb331f83953cd9e7eb815361ff78f01c3b40a5b62276c64a" exitCode=0
Jul 30 01:16:07.925841 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:07.925694    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9" event={"ID":"b8c75281-d1d4-45f2-b625-774a67216bb4","Type":"ContainerDied","Data":"c5cf606256ee52c4eb331f83953cd9e7eb815361ff78f01c3b40a5b62276c64a"}
Jul 30 01:16:07.928188 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:07.928054    2546 generic.go:356] "Generic (PLEG): container finished" podID="da0227be-d5f6-46e5-8ccd-dbbd72632683" containerID="6bf8c7df9a2ff0651e69ab75e665236d11cc86515a92cd3d7f9a60f290cee66b" exitCode=0
Jul 30 01:16:07.928327 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:07.928206    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp" event={"ID":"da0227be-d5f6-46e5-8ccd-dbbd72632683","Type":"ContainerDied","Data":"6bf8c7df9a2ff0651e69ab75e665236d11cc86515a92cd3d7f9a60f290cee66b"}
Jul 30 01:16:07.930144 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:07.930086    2546 generic.go:356] "Generic (PLEG): container finished" podID="ee5f4e73-b40a-4d51-80ed-826c65767385" containerID="c2879a93ce2773b2314ba26511c1dea2c82a60463dcf345e8fb0046a58d37acf" exitCode=0
Jul 30 01:16:07.930144 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:07.930132    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp" event={"ID":"ee5f4e73-b40a-4d51-80ed-826c65767385","Type":"ContainerDied","Data":"c2879a93ce2773b2314ba26511c1dea2c82a60463dcf345e8fb0046a58d37acf"}
Jul 30 01:16:07.996893 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:07.996863    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9"
Jul 30 01:16:08.058271 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.058240    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp"
Jul 30 01:16:08.072585 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.072561    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp"
Jul 30 01:16:08.121770 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.121740    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l89br\" (UniqueName: \"kubernetes.io/projected/b8c75281-d1d4-45f2-b625-774a67216bb4-kube-api-access-l89br\") pod \"b8c75281-d1d4-45f2-b625-774a67216bb4\" (UID: \"b8c75281-d1d4-45f2-b625-774a67216bb4\") "
Jul 30 01:16:08.121921 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.121809    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8c75281-d1d4-45f2-b625-774a67216bb4-plugins-config-volume\") pod \"b8c75281-d1d4-45f2-b625-774a67216bb4\" (UID: \"b8c75281-d1d4-45f2-b625-774a67216bb4\") "
Jul 30 01:16:08.122194 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.122168    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8c75281-d1d4-45f2-b625-774a67216bb4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b8c75281-d1d4-45f2-b625-774a67216bb4" (UID: "b8c75281-d1d4-45f2-b625-774a67216bb4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:16:08.123892 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.123865    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8c75281-d1d4-45f2-b625-774a67216bb4-kube-api-access-l89br" (OuterVolumeSpecName: "kube-api-access-l89br") pod "b8c75281-d1d4-45f2-b625-774a67216bb4" (UID: "b8c75281-d1d4-45f2-b625-774a67216bb4"). InnerVolumeSpecName "kube-api-access-l89br". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:16:08.222280 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.222242    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da0227be-d5f6-46e5-8ccd-dbbd72632683-openshift-service-ca\") pod \"da0227be-d5f6-46e5-8ccd-dbbd72632683\" (UID: \"da0227be-d5f6-46e5-8ccd-dbbd72632683\") "
Jul 30 01:16:08.222479 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.222304    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zd6n\" (UniqueName: \"kubernetes.io/projected/ee5f4e73-b40a-4d51-80ed-826c65767385-kube-api-access-2zd6n\") pod \"ee5f4e73-b40a-4d51-80ed-826c65767385\" (UID: \"ee5f4e73-b40a-4d51-80ed-826c65767385\") "
Jul 30 01:16:08.222479 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.222423    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sscmd\" (UniqueName: \"kubernetes.io/projected/da0227be-d5f6-46e5-8ccd-dbbd72632683-kube-api-access-sscmd\") pod \"da0227be-d5f6-46e5-8ccd-dbbd72632683\" (UID: \"da0227be-d5f6-46e5-8ccd-dbbd72632683\") "
Jul 30 01:16:08.222479 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.222468    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee5f4e73-b40a-4d51-80ed-826c65767385-plugins-config-volume\") pod \"ee5f4e73-b40a-4d51-80ed-826c65767385\" (UID: \"ee5f4e73-b40a-4d51-80ed-826c65767385\") "
Jul 30 01:16:08.222706 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.222514    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da0227be-d5f6-46e5-8ccd-dbbd72632683-maas-api-tls\") pod \"da0227be-d5f6-46e5-8ccd-dbbd72632683\" (UID: \"da0227be-d5f6-46e5-8ccd-dbbd72632683\") "
Jul 30 01:16:08.222808 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.222780    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da0227be-d5f6-46e5-8ccd-dbbd72632683-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "da0227be-d5f6-46e5-8ccd-dbbd72632683" (UID: "da0227be-d5f6-46e5-8ccd-dbbd72632683"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:16:08.222871 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.222795    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l89br\" (UniqueName: \"kubernetes.io/projected/b8c75281-d1d4-45f2-b625-774a67216bb4-kube-api-access-l89br\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:16:08.222871 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.222841    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8c75281-d1d4-45f2-b625-774a67216bb4-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:16:08.222871 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.222866    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee5f4e73-b40a-4d51-80ed-826c65767385-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ee5f4e73-b40a-4d51-80ed-826c65767385" (UID: "ee5f4e73-b40a-4d51-80ed-826c65767385"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:16:08.224594 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.224566    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee5f4e73-b40a-4d51-80ed-826c65767385-kube-api-access-2zd6n" (OuterVolumeSpecName: "kube-api-access-2zd6n") pod "ee5f4e73-b40a-4d51-80ed-826c65767385" (UID: "ee5f4e73-b40a-4d51-80ed-826c65767385"). InnerVolumeSpecName "kube-api-access-2zd6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:16:08.224739 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.224719    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da0227be-d5f6-46e5-8ccd-dbbd72632683-kube-api-access-sscmd" (OuterVolumeSpecName: "kube-api-access-sscmd") pod "da0227be-d5f6-46e5-8ccd-dbbd72632683" (UID: "da0227be-d5f6-46e5-8ccd-dbbd72632683"). InnerVolumeSpecName "kube-api-access-sscmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:16:08.224801 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.224735    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da0227be-d5f6-46e5-8ccd-dbbd72632683-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "da0227be-d5f6-46e5-8ccd-dbbd72632683" (UID: "da0227be-d5f6-46e5-8ccd-dbbd72632683"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:16:08.323506 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.323458    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zd6n\" (UniqueName: \"kubernetes.io/projected/ee5f4e73-b40a-4d51-80ed-826c65767385-kube-api-access-2zd6n\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:16:08.323506 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.323497    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sscmd\" (UniqueName: \"kubernetes.io/projected/da0227be-d5f6-46e5-8ccd-dbbd72632683-kube-api-access-sscmd\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:16:08.323506 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.323508    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee5f4e73-b40a-4d51-80ed-826c65767385-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:16:08.323777 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.323547    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da0227be-d5f6-46e5-8ccd-dbbd72632683-maas-api-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:16:08.323777 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.323560    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da0227be-d5f6-46e5-8ccd-dbbd72632683-openshift-service-ca\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:16:08.937619 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.937538    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp"
Jul 30 01:16:08.937619 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.937556    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp" event={"ID":"da0227be-d5f6-46e5-8ccd-dbbd72632683","Type":"ContainerDied","Data":"fddb4df3be6255f783d90d097a932bc49307e2c539bfef73621be730119297a7"}
Jul 30 01:16:08.938180 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.937629    2546 scope.go:117] "RemoveContainer" containerID="6bf8c7df9a2ff0651e69ab75e665236d11cc86515a92cd3d7f9a60f290cee66b"
Jul 30 01:16:08.939092 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.939071    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp"
Jul 30 01:16:08.939182 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.939101    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp" event={"ID":"ee5f4e73-b40a-4d51-80ed-826c65767385","Type":"ContainerDied","Data":"4de941133a8f334836bdd809589b96dd2f6a768b615e48c42f3ad6e4d724e784"}
Jul 30 01:16:08.948978 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.948904    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9"
Jul 30 01:16:08.950401 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.949324    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9" event={"ID":"b8c75281-d1d4-45f2-b625-774a67216bb4","Type":"ContainerDied","Data":"b49b8c2d69cd74f3f425b5eb01c00728c552dd294f90d7b8f09a5259ef48d5c8"}
Jul 30 01:16:08.952105 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.952064    2546 scope.go:117] "RemoveContainer" containerID="c2879a93ce2773b2314ba26511c1dea2c82a60463dcf345e8fb0046a58d37acf"
Jul 30 01:16:08.961488 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.961468    2546 scope.go:117] "RemoveContainer" containerID="c5cf606256ee52c4eb331f83953cd9e7eb815361ff78f01c3b40a5b62276c64a"
Jul 30 01:16:08.974095 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.974070    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp"]
Jul 30 01:16:08.983489 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:08.983461    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-76aaf06c-7d697b746b-r5jbp"]
Jul 30 01:16:09.003398 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:09.003339    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9"]
Jul 30 01:16:09.010559 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:09.010511    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-76aaf06c-bb948fdb8-lcws9"]
Jul 30 01:16:09.030994 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:09.030967    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp"]
Jul 30 01:16:09.040181 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:09.040152    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-76aaf06c-b9cbd74d4-6l6fp"]
Jul 30 01:16:09.358537 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:09.358490    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8c75281-d1d4-45f2-b625-774a67216bb4" path="/var/lib/kubelet/pods/b8c75281-d1d4-45f2-b625-774a67216bb4/volumes"
Jul 30 01:16:09.358927 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:09.358912    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da0227be-d5f6-46e5-8ccd-dbbd72632683" path="/var/lib/kubelet/pods/da0227be-d5f6-46e5-8ccd-dbbd72632683/volumes"
Jul 30 01:16:09.359237 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:09.359225    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee5f4e73-b40a-4d51-80ed-826c65767385" path="/var/lib/kubelet/pods/ee5f4e73-b40a-4d51-80ed-826c65767385/volumes"
Jul 30 01:16:11.451100 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:11.451062    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm"]
Jul 30 01:16:11.451684 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:11.451667    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da0227be-d5f6-46e5-8ccd-dbbd72632683" containerName="maas-api"
Jul 30 01:16:11.451735 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:11.451687    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="da0227be-d5f6-46e5-8ccd-dbbd72632683" containerName="maas-api"
Jul 30 01:16:11.451735 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:11.451709    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8c75281-d1d4-45f2-b625-774a67216bb4" containerName="payload-processing"
Jul 30 01:16:11.451735 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:11.451719    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8c75281-d1d4-45f2-b625-774a67216bb4" containerName="payload-processing"
Jul 30 01:16:11.451834 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:11.451741    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee5f4e73-b40a-4d51-80ed-826c65767385" containerName="payload-pre-processing"
Jul 30 01:16:11.451834 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:11.451750    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee5f4e73-b40a-4d51-80ed-826c65767385" containerName="payload-pre-processing"
Jul 30 01:16:11.451899 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:11.451863    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee5f4e73-b40a-4d51-80ed-826c65767385" containerName="payload-pre-processing"
Jul 30 01:16:11.451899 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:11.451875    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8c75281-d1d4-45f2-b625-774a67216bb4" containerName="payload-processing"
Jul 30 01:16:11.451899 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:11.451890    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="da0227be-d5f6-46e5-8ccd-dbbd72632683" containerName="maas-api"
Jul 30 01:16:11.457079 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:11.457054    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm"
Jul 30 01:16:11.460075 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:11.460054    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-a6605e44-serving-cert\""
Jul 30 01:16:11.466867 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:11.466831    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm"]
Jul 30 01:16:11.555830 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:11.555794    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4fc4bf3-ec6e-4228-8ea4-df35d40503c6-openshift-service-ca\") pod \"maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm\" (UID: \"f4fc4bf3-ec6e-4228-8ea4-df35d40503c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm"
Jul 30 01:16:11.556004 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:11.555868    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4fc4bf3-ec6e-4228-8ea4-df35d40503c6-maas-api-tls\") pod \"maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm\" (UID: \"f4fc4bf3-ec6e-4228-8ea4-df35d40503c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm"
Jul 30 01:16:11.556004 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:11.555930    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgvlc\" (UniqueName: \"kubernetes.io/projected/f4fc4bf3-ec6e-4228-8ea4-df35d40503c6-kube-api-access-kgvlc\") pod \"maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm\" (UID: \"f4fc4bf3-ec6e-4228-8ea4-df35d40503c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm"
Jul 30 01:16:11.657381 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:11.657341    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4fc4bf3-ec6e-4228-8ea4-df35d40503c6-openshift-service-ca\") pod \"maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm\" (UID: \"f4fc4bf3-ec6e-4228-8ea4-df35d40503c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm"
Jul 30 01:16:11.657591 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:11.657410    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4fc4bf3-ec6e-4228-8ea4-df35d40503c6-maas-api-tls\") pod \"maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm\" (UID: \"f4fc4bf3-ec6e-4228-8ea4-df35d40503c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm"
Jul 30 01:16:11.657591 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:11.657443    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgvlc\" (UniqueName: \"kubernetes.io/projected/f4fc4bf3-ec6e-4228-8ea4-df35d40503c6-kube-api-access-kgvlc\") pod \"maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm\" (UID: \"f4fc4bf3-ec6e-4228-8ea4-df35d40503c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm"
Jul 30 01:16:11.657591 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:16:11.657517    2546 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-serving-cert: secret "maas-api-e2e-migrate-a6605e44-serving-cert" not found
Jul 30 01:16:11.657723 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:16:11.657640    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4fc4bf3-ec6e-4228-8ea4-df35d40503c6-maas-api-tls podName:f4fc4bf3-ec6e-4228-8ea4-df35d40503c6 nodeName:}" failed. No retries permitted until 2026-07-30 01:16:12.157615637 +0000 UTC m=+1959.380349440 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f4fc4bf3-ec6e-4228-8ea4-df35d40503c6-maas-api-tls") pod "maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm" (UID: "f4fc4bf3-ec6e-4228-8ea4-df35d40503c6") : secret "maas-api-e2e-migrate-a6605e44-serving-cert" not found
Jul 30 01:16:11.658166 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:11.658148    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4fc4bf3-ec6e-4228-8ea4-df35d40503c6-openshift-service-ca\") pod \"maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm\" (UID: \"f4fc4bf3-ec6e-4228-8ea4-df35d40503c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm"
Jul 30 01:16:11.669122 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:11.669103    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgvlc\" (UniqueName: \"kubernetes.io/projected/f4fc4bf3-ec6e-4228-8ea4-df35d40503c6-kube-api-access-kgvlc\") pod \"maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm\" (UID: \"f4fc4bf3-ec6e-4228-8ea4-df35d40503c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm"
Jul 30 01:16:12.080306 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.080271    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m"]
Jul 30 01:16:12.083939 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.083924    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m"
Jul 30 01:16:12.086574 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.086549    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-a6605e44\""
Jul 30 01:16:12.086712 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.086605    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-a6605e44-dockercfg-jrzcs\""
Jul 30 01:16:12.094774 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.094747    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m"]
Jul 30 01:16:12.123577 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.123543    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq"]
Jul 30 01:16:12.127288 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.127268    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq"
Jul 30 01:16:12.134777 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.134749    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq"]
Jul 30 01:16:12.162273 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.162218    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4fc4bf3-ec6e-4228-8ea4-df35d40503c6-maas-api-tls\") pod \"maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm\" (UID: \"f4fc4bf3-ec6e-4228-8ea4-df35d40503c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm"
Jul 30 01:16:12.164694 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.164669    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4fc4bf3-ec6e-4228-8ea4-df35d40503c6-maas-api-tls\") pod \"maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm\" (UID: \"f4fc4bf3-ec6e-4228-8ea4-df35d40503c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm"
Jul 30 01:16:12.263201 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.263158    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fa652e7-080f-4b77-9489-e677ff09dd9b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq\" (UID: \"6fa652e7-080f-4b77-9489-e677ff09dd9b\") " pod="openshift-ingress/payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq"
Jul 30 01:16:12.263201 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.263207    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lpwz\" (UniqueName: \"kubernetes.io/projected/6fa652e7-080f-4b77-9489-e677ff09dd9b-kube-api-access-6lpwz\") pod \"payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq\" (UID: \"6fa652e7-080f-4b77-9489-e677ff09dd9b\") " pod="openshift-ingress/payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq"
Jul 30 01:16:12.263412 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.263245    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn2mg\" (UniqueName: \"kubernetes.io/projected/1238c2d5-14bb-4c52-ac8f-586d12769d84-kube-api-access-jn2mg\") pod \"payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m\" (UID: \"1238c2d5-14bb-4c52-ac8f-586d12769d84\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m"
Jul 30 01:16:12.263412 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.263280    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1238c2d5-14bb-4c52-ac8f-586d12769d84-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m\" (UID: \"1238c2d5-14bb-4c52-ac8f-586d12769d84\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m"
Jul 30 01:16:12.364015 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.363932    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fa652e7-080f-4b77-9489-e677ff09dd9b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq\" (UID: \"6fa652e7-080f-4b77-9489-e677ff09dd9b\") " pod="openshift-ingress/payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq"
Jul 30 01:16:12.364015 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.363974    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lpwz\" (UniqueName: \"kubernetes.io/projected/6fa652e7-080f-4b77-9489-e677ff09dd9b-kube-api-access-6lpwz\") pod \"payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq\" (UID: \"6fa652e7-080f-4b77-9489-e677ff09dd9b\") " pod="openshift-ingress/payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq"
Jul 30 01:16:12.364015 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.363999    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn2mg\" (UniqueName: \"kubernetes.io/projected/1238c2d5-14bb-4c52-ac8f-586d12769d84-kube-api-access-jn2mg\") pod \"payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m\" (UID: \"1238c2d5-14bb-4c52-ac8f-586d12769d84\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m"
Jul 30 01:16:12.364234 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.364022    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1238c2d5-14bb-4c52-ac8f-586d12769d84-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m\" (UID: \"1238c2d5-14bb-4c52-ac8f-586d12769d84\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m"
Jul 30 01:16:12.364717 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.364689    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1238c2d5-14bb-4c52-ac8f-586d12769d84-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m\" (UID: \"1238c2d5-14bb-4c52-ac8f-586d12769d84\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m"
Jul 30 01:16:12.364838 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.364821    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fa652e7-080f-4b77-9489-e677ff09dd9b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq\" (UID: \"6fa652e7-080f-4b77-9489-e677ff09dd9b\") " pod="openshift-ingress/payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq"
Jul 30 01:16:12.369678 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.369657    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm"
Jul 30 01:16:12.372042 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.372021    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn2mg\" (UniqueName: \"kubernetes.io/projected/1238c2d5-14bb-4c52-ac8f-586d12769d84-kube-api-access-jn2mg\") pod \"payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m\" (UID: \"1238c2d5-14bb-4c52-ac8f-586d12769d84\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m"
Jul 30 01:16:12.372244 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.372213    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lpwz\" (UniqueName: \"kubernetes.io/projected/6fa652e7-080f-4b77-9489-e677ff09dd9b-kube-api-access-6lpwz\") pod \"payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq\" (UID: \"6fa652e7-080f-4b77-9489-e677ff09dd9b\") " pod="openshift-ingress/payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq"
Jul 30 01:16:12.397302 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.397273    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m"
Jul 30 01:16:12.439613 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.439574    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq"
Jul 30 01:16:12.605143 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.605116    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq"]
Jul 30 01:16:12.607389 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:16:12.607362    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fa652e7_080f_4b77_9489_e677ff09dd9b.slice/crio-adaccd792bed9aed2c355df03d3a4e5458bd6ba6364906e0a2a431d608306e3c WatchSource:0}: Error finding container adaccd792bed9aed2c355df03d3a4e5458bd6ba6364906e0a2a431d608306e3c: Status 404 returned error can't find the container with id adaccd792bed9aed2c355df03d3a4e5458bd6ba6364906e0a2a431d608306e3c
Jul 30 01:16:12.747310 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.747283    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm"]
Jul 30 01:16:12.749280 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:16:12.749248    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4fc4bf3_ec6e_4228_8ea4_df35d40503c6.slice/crio-5ff93d76d23b7b4a255f2247ac6fc65fc4ff924e1f1ed930b10a1c4da44e226c WatchSource:0}: Error finding container 5ff93d76d23b7b4a255f2247ac6fc65fc4ff924e1f1ed930b10a1c4da44e226c: Status 404 returned error can't find the container with id 5ff93d76d23b7b4a255f2247ac6fc65fc4ff924e1f1ed930b10a1c4da44e226c
Jul 30 01:16:12.777088 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.777068    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m"]
Jul 30 01:16:12.779085 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:16:12.779053    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1238c2d5_14bb_4c52_ac8f_586d12769d84.slice/crio-67be81d3df1c780fe079ebdf52c5a5cca59c368bd91c3f0112d45728e0481f7d WatchSource:0}: Error finding container 67be81d3df1c780fe079ebdf52c5a5cca59c368bd91c3f0112d45728e0481f7d: Status 404 returned error can't find the container with id 67be81d3df1c780fe079ebdf52c5a5cca59c368bd91c3f0112d45728e0481f7d
Jul 30 01:16:12.973308 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.973275    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm" event={"ID":"f4fc4bf3-ec6e-4228-8ea4-df35d40503c6","Type":"ContainerStarted","Data":"5ff93d76d23b7b4a255f2247ac6fc65fc4ff924e1f1ed930b10a1c4da44e226c"}
Jul 30 01:16:12.974791 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.974767    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq" event={"ID":"6fa652e7-080f-4b77-9489-e677ff09dd9b","Type":"ContainerStarted","Data":"1dd989c08a29e6afce488a53cd309868f6e777df503aab507541118485487fa4"}
Jul 30 01:16:12.974949 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.974821    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq"
Jul 30 01:16:12.974949 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.974844    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq" event={"ID":"6fa652e7-080f-4b77-9489-e677ff09dd9b","Type":"ContainerStarted","Data":"adaccd792bed9aed2c355df03d3a4e5458bd6ba6364906e0a2a431d608306e3c"}
Jul 30 01:16:12.976271 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.976250    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m" event={"ID":"1238c2d5-14bb-4c52-ac8f-586d12769d84","Type":"ContainerStarted","Data":"32a2f82d33f13dcf402cc3301524ba41367149f08cebb3c7f2392edceb8874b6"}
Jul 30 01:16:12.976343 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.976295    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m" event={"ID":"1238c2d5-14bb-4c52-ac8f-586d12769d84","Type":"ContainerStarted","Data":"67be81d3df1c780fe079ebdf52c5a5cca59c368bd91c3f0112d45728e0481f7d"}
Jul 30 01:16:12.976419 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.976387    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m"
Jul 30 01:16:12.994037 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:12.993978    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq" podStartSLOduration=0.993957009 podStartE2EDuration="993.957009ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:16:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:16:12.991214267 +0000 UTC m=+1960.213948083" watchObservedRunningTime="2026-07-30 01:16:12.993957009 +0000 UTC m=+1960.216690831"
Jul 30 01:16:13.009089 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:13.009044    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m" podStartSLOduration=1.00903001 podStartE2EDuration="1.00903001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:16:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:16:13.007433664 +0000 UTC m=+1960.230167486" watchObservedRunningTime="2026-07-30 01:16:13.00903001 +0000 UTC m=+1960.231763827"
Jul 30 01:16:13.982797 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:13.982761    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm" event={"ID":"f4fc4bf3-ec6e-4228-8ea4-df35d40503c6","Type":"ContainerStarted","Data":"03dfc52e5958872b374bb34e48cb8d2378ab35b0d87a06849b2d40abf77a6ecb"}
Jul 30 01:16:13.983286 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:13.983131    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm"
Jul 30 01:16:14.000834 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:14.000788    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm" podStartSLOduration=2.633685231 podStartE2EDuration="3.000772789s" totalImagesPullingTime="367.087558ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:16:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:16:13.998738968 +0000 UTC m=+1961.221472782" watchObservedRunningTime="2026-07-30 01:16:14.000772789 +0000 UTC m=+1961.223506607"
Jul 30 01:16:19.996728 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:19.996696    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm"
Jul 30 01:16:23.984975 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:23.984936    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq"
Jul 30 01:16:23.987374 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:23.985025    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m"
Jul 30 01:16:42.774929 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:42.774886    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm"]
Jul 30 01:16:42.775434 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:42.775166    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm" podUID="f4fc4bf3-ec6e-4228-8ea4-df35d40503c6" containerName="maas-api" containerID="cri-o://03dfc52e5958872b374bb34e48cb8d2378ab35b0d87a06849b2d40abf77a6ecb" gracePeriod=30
Jul 30 01:16:42.832665 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:42.832633    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq"]
Jul 30 01:16:42.832915 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:42.832883    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq" podUID="6fa652e7-080f-4b77-9489-e677ff09dd9b" containerName="payload-processing" containerID="cri-o://1dd989c08a29e6afce488a53cd309868f6e777df503aab507541118485487fa4" gracePeriod=30
Jul 30 01:16:42.840313 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:42.840266    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m"]
Jul 30 01:16:42.840541 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:42.840491    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m" podUID="1238c2d5-14bb-4c52-ac8f-586d12769d84" containerName="payload-pre-processing" containerID="cri-o://32a2f82d33f13dcf402cc3301524ba41367149f08cebb3c7f2392edceb8874b6" gracePeriod=30
Jul 30 01:16:43.127200 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.127170    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm"
Jul 30 01:16:43.144991 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.144963    2546 generic.go:356] "Generic (PLEG): container finished" podID="f4fc4bf3-ec6e-4228-8ea4-df35d40503c6" containerID="03dfc52e5958872b374bb34e48cb8d2378ab35b0d87a06849b2d40abf77a6ecb" exitCode=0
Jul 30 01:16:43.145161 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.145019    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm"
Jul 30 01:16:43.145161 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.145049    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm" event={"ID":"f4fc4bf3-ec6e-4228-8ea4-df35d40503c6","Type":"ContainerDied","Data":"03dfc52e5958872b374bb34e48cb8d2378ab35b0d87a06849b2d40abf77a6ecb"}
Jul 30 01:16:43.145161 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.145101    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm" event={"ID":"f4fc4bf3-ec6e-4228-8ea4-df35d40503c6","Type":"ContainerDied","Data":"5ff93d76d23b7b4a255f2247ac6fc65fc4ff924e1f1ed930b10a1c4da44e226c"}
Jul 30 01:16:43.145161 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.145133    2546 scope.go:117] "RemoveContainer" containerID="03dfc52e5958872b374bb34e48cb8d2378ab35b0d87a06849b2d40abf77a6ecb"
Jul 30 01:16:43.147808 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.147725    2546 generic.go:356] "Generic (PLEG): container finished" podID="6fa652e7-080f-4b77-9489-e677ff09dd9b" containerID="1dd989c08a29e6afce488a53cd309868f6e777df503aab507541118485487fa4" exitCode=0
Jul 30 01:16:43.148043 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.147827    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq" event={"ID":"6fa652e7-080f-4b77-9489-e677ff09dd9b","Type":"ContainerDied","Data":"1dd989c08a29e6afce488a53cd309868f6e777df503aab507541118485487fa4"}
Jul 30 01:16:43.154124 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.154098    2546 generic.go:356] "Generic (PLEG): container finished" podID="1238c2d5-14bb-4c52-ac8f-586d12769d84" containerID="32a2f82d33f13dcf402cc3301524ba41367149f08cebb3c7f2392edceb8874b6" exitCode=0
Jul 30 01:16:43.154254 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.154171    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m" event={"ID":"1238c2d5-14bb-4c52-ac8f-586d12769d84","Type":"ContainerDied","Data":"32a2f82d33f13dcf402cc3301524ba41367149f08cebb3c7f2392edceb8874b6"}
Jul 30 01:16:43.159053 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.159033    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m"
Jul 30 01:16:43.159509 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.159490    2546 scope.go:117] "RemoveContainer" containerID="03dfc52e5958872b374bb34e48cb8d2378ab35b0d87a06849b2d40abf77a6ecb"
Jul 30 01:16:43.159821 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:16:43.159802    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03dfc52e5958872b374bb34e48cb8d2378ab35b0d87a06849b2d40abf77a6ecb\": container with ID starting with 03dfc52e5958872b374bb34e48cb8d2378ab35b0d87a06849b2d40abf77a6ecb not found: ID does not exist" containerID="03dfc52e5958872b374bb34e48cb8d2378ab35b0d87a06849b2d40abf77a6ecb"
Jul 30 01:16:43.159896 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.159831    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03dfc52e5958872b374bb34e48cb8d2378ab35b0d87a06849b2d40abf77a6ecb"} err="failed to get container status \"03dfc52e5958872b374bb34e48cb8d2378ab35b0d87a06849b2d40abf77a6ecb\": rpc error: code = NotFound desc = could not find container \"03dfc52e5958872b374bb34e48cb8d2378ab35b0d87a06849b2d40abf77a6ecb\": container with ID starting with 03dfc52e5958872b374bb34e48cb8d2378ab35b0d87a06849b2d40abf77a6ecb not found: ID does not exist"
Jul 30 01:16:43.161735 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.161720    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq"
Jul 30 01:16:43.225210 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.225176    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fa652e7-080f-4b77-9489-e677ff09dd9b-plugins-config-volume\") pod \"6fa652e7-080f-4b77-9489-e677ff09dd9b\" (UID: \"6fa652e7-080f-4b77-9489-e677ff09dd9b\") "
Jul 30 01:16:43.225354 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.225220    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgvlc\" (UniqueName: \"kubernetes.io/projected/f4fc4bf3-ec6e-4228-8ea4-df35d40503c6-kube-api-access-kgvlc\") pod \"f4fc4bf3-ec6e-4228-8ea4-df35d40503c6\" (UID: \"f4fc4bf3-ec6e-4228-8ea4-df35d40503c6\") "
Jul 30 01:16:43.225354 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.225239    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jn2mg\" (UniqueName: \"kubernetes.io/projected/1238c2d5-14bb-4c52-ac8f-586d12769d84-kube-api-access-jn2mg\") pod \"1238c2d5-14bb-4c52-ac8f-586d12769d84\" (UID: \"1238c2d5-14bb-4c52-ac8f-586d12769d84\") "
Jul 30 01:16:43.225354 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.225320    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4fc4bf3-ec6e-4228-8ea4-df35d40503c6-maas-api-tls\") pod \"f4fc4bf3-ec6e-4228-8ea4-df35d40503c6\" (UID: \"f4fc4bf3-ec6e-4228-8ea4-df35d40503c6\") "
Jul 30 01:16:43.225511 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.225374    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lpwz\" (UniqueName: \"kubernetes.io/projected/6fa652e7-080f-4b77-9489-e677ff09dd9b-kube-api-access-6lpwz\") pod \"6fa652e7-080f-4b77-9489-e677ff09dd9b\" (UID: \"6fa652e7-080f-4b77-9489-e677ff09dd9b\") "
Jul 30 01:16:43.225511 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.225419    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1238c2d5-14bb-4c52-ac8f-586d12769d84-plugins-config-volume\") pod \"1238c2d5-14bb-4c52-ac8f-586d12769d84\" (UID: \"1238c2d5-14bb-4c52-ac8f-586d12769d84\") "
Jul 30 01:16:43.225511 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.225458    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4fc4bf3-ec6e-4228-8ea4-df35d40503c6-openshift-service-ca\") pod \"f4fc4bf3-ec6e-4228-8ea4-df35d40503c6\" (UID: \"f4fc4bf3-ec6e-4228-8ea4-df35d40503c6\") "
Jul 30 01:16:43.225798 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.225606    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fa652e7-080f-4b77-9489-e677ff09dd9b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6fa652e7-080f-4b77-9489-e677ff09dd9b" (UID: "6fa652e7-080f-4b77-9489-e677ff09dd9b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:16:43.225798 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.225746    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fa652e7-080f-4b77-9489-e677ff09dd9b-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:16:43.225905 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.225855    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1238c2d5-14bb-4c52-ac8f-586d12769d84-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1238c2d5-14bb-4c52-ac8f-586d12769d84" (UID: "1238c2d5-14bb-4c52-ac8f-586d12769d84"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:16:43.226057 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.226031    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4fc4bf3-ec6e-4228-8ea4-df35d40503c6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f4fc4bf3-ec6e-4228-8ea4-df35d40503c6" (UID: "f4fc4bf3-ec6e-4228-8ea4-df35d40503c6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:16:43.227512 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.227483    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1238c2d5-14bb-4c52-ac8f-586d12769d84-kube-api-access-jn2mg" (OuterVolumeSpecName: "kube-api-access-jn2mg") pod "1238c2d5-14bb-4c52-ac8f-586d12769d84" (UID: "1238c2d5-14bb-4c52-ac8f-586d12769d84"). InnerVolumeSpecName "kube-api-access-jn2mg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:16:43.227512 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.227493    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4fc4bf3-ec6e-4228-8ea4-df35d40503c6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f4fc4bf3-ec6e-4228-8ea4-df35d40503c6" (UID: "f4fc4bf3-ec6e-4228-8ea4-df35d40503c6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:16:43.227813 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.227791    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fa652e7-080f-4b77-9489-e677ff09dd9b-kube-api-access-6lpwz" (OuterVolumeSpecName: "kube-api-access-6lpwz") pod "6fa652e7-080f-4b77-9489-e677ff09dd9b" (UID: "6fa652e7-080f-4b77-9489-e677ff09dd9b"). InnerVolumeSpecName "kube-api-access-6lpwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:16:43.227813 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.227802    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4fc4bf3-ec6e-4228-8ea4-df35d40503c6-kube-api-access-kgvlc" (OuterVolumeSpecName: "kube-api-access-kgvlc") pod "f4fc4bf3-ec6e-4228-8ea4-df35d40503c6" (UID: "f4fc4bf3-ec6e-4228-8ea4-df35d40503c6"). InnerVolumeSpecName "kube-api-access-kgvlc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:16:43.327987 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.327908    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4fc4bf3-ec6e-4228-8ea4-df35d40503c6-maas-api-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:16:43.327987 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.327944    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lpwz\" (UniqueName: \"kubernetes.io/projected/6fa652e7-080f-4b77-9489-e677ff09dd9b-kube-api-access-6lpwz\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:16:43.327987 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.327967    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1238c2d5-14bb-4c52-ac8f-586d12769d84-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:16:43.327987 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.327984    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4fc4bf3-ec6e-4228-8ea4-df35d40503c6-openshift-service-ca\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:16:43.328283 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.327998    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgvlc\" (UniqueName: \"kubernetes.io/projected/f4fc4bf3-ec6e-4228-8ea4-df35d40503c6-kube-api-access-kgvlc\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:16:43.328283 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.328011    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jn2mg\" (UniqueName: \"kubernetes.io/projected/1238c2d5-14bb-4c52-ac8f-586d12769d84-kube-api-access-jn2mg\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:16:43.481727 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.481693    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm"]
Jul 30 01:16:43.489175 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:43.489142    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-a6605e44-5b6db5764d-bckqm"]
Jul 30 01:16:44.162460 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:44.162366    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq" event={"ID":"6fa652e7-080f-4b77-9489-e677ff09dd9b","Type":"ContainerDied","Data":"adaccd792bed9aed2c355df03d3a4e5458bd6ba6364906e0a2a431d608306e3c"}
Jul 30 01:16:44.162460 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:44.162428    2546 scope.go:117] "RemoveContainer" containerID="1dd989c08a29e6afce488a53cd309868f6e777df503aab507541118485487fa4"
Jul 30 01:16:44.162460 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:44.162376    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq"
Jul 30 01:16:44.163811 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:44.163769    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m"
Jul 30 01:16:44.163811 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:44.163797    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m" event={"ID":"1238c2d5-14bb-4c52-ac8f-586d12769d84","Type":"ContainerDied","Data":"67be81d3df1c780fe079ebdf52c5a5cca59c368bd91c3f0112d45728e0481f7d"}
Jul 30 01:16:44.171297 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:44.171281    2546 scope.go:117] "RemoveContainer" containerID="32a2f82d33f13dcf402cc3301524ba41367149f08cebb3c7f2392edceb8874b6"
Jul 30 01:16:44.184503 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:44.184481    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m"]
Jul 30 01:16:44.189008 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:44.188982    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a6605e44-5655bc988d-b7g4m"]
Jul 30 01:16:44.203337 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:44.203315    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq"]
Jul 30 01:16:44.207937 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:44.207918    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a6605e44-b759f5d5-xw5qq"]
Jul 30 01:16:45.358952 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:45.358922    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1238c2d5-14bb-4c52-ac8f-586d12769d84" path="/var/lib/kubelet/pods/1238c2d5-14bb-4c52-ac8f-586d12769d84/volumes"
Jul 30 01:16:45.359322 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:45.359286    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fa652e7-080f-4b77-9489-e677ff09dd9b" path="/var/lib/kubelet/pods/6fa652e7-080f-4b77-9489-e677ff09dd9b/volumes"
Jul 30 01:16:45.359641 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:45.359627    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4fc4bf3-ec6e-4228-8ea4-df35d40503c6" path="/var/lib/kubelet/pods/f4fc4bf3-ec6e-4228-8ea4-df35d40503c6/volumes"
Jul 30 01:16:49.748832 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:49.748800    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n"]
Jul 30 01:16:49.749222 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:49.749184    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4fc4bf3-ec6e-4228-8ea4-df35d40503c6" containerName="maas-api"
Jul 30 01:16:49.749222 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:49.749197    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4fc4bf3-ec6e-4228-8ea4-df35d40503c6" containerName="maas-api"
Jul 30 01:16:49.749222 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:49.749215    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1238c2d5-14bb-4c52-ac8f-586d12769d84" containerName="payload-pre-processing"
Jul 30 01:16:49.749222 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:49.749221    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="1238c2d5-14bb-4c52-ac8f-586d12769d84" containerName="payload-pre-processing"
Jul 30 01:16:49.749370 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:49.749238    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fa652e7-080f-4b77-9489-e677ff09dd9b" containerName="payload-processing"
Jul 30 01:16:49.749370 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:49.749244    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fa652e7-080f-4b77-9489-e677ff09dd9b" containerName="payload-processing"
Jul 30 01:16:49.749370 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:49.749295    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4fc4bf3-ec6e-4228-8ea4-df35d40503c6" containerName="maas-api"
Jul 30 01:16:49.749370 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:49.749306    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="1238c2d5-14bb-4c52-ac8f-586d12769d84" containerName="payload-pre-processing"
Jul 30 01:16:49.749370 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:49.749314    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fa652e7-080f-4b77-9489-e677ff09dd9b" containerName="payload-processing"
Jul 30 01:16:49.754131 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:49.754111    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n"
Jul 30 01:16:49.756697 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:49.756673    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-095ac0a0-serving-cert\""
Jul 30 01:16:49.764024 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:49.764002    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n"]
Jul 30 01:16:49.883613 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:49.883579    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/437f21e3-0aba-4826-bfcf-2ceffb0d127a-maas-api-tls\") pod \"maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n\" (UID: \"437f21e3-0aba-4826-bfcf-2ceffb0d127a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n"
Jul 30 01:16:49.883785 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:49.883634    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/437f21e3-0aba-4826-bfcf-2ceffb0d127a-openshift-service-ca\") pod \"maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n\" (UID: \"437f21e3-0aba-4826-bfcf-2ceffb0d127a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n"
Jul 30 01:16:49.883785 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:49.883671    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhmq7\" (UniqueName: \"kubernetes.io/projected/437f21e3-0aba-4826-bfcf-2ceffb0d127a-kube-api-access-bhmq7\") pod \"maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n\" (UID: \"437f21e3-0aba-4826-bfcf-2ceffb0d127a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n"
Jul 30 01:16:49.984730 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:49.984693    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/437f21e3-0aba-4826-bfcf-2ceffb0d127a-maas-api-tls\") pod \"maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n\" (UID: \"437f21e3-0aba-4826-bfcf-2ceffb0d127a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n"
Jul 30 01:16:49.984901 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:49.984742    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/437f21e3-0aba-4826-bfcf-2ceffb0d127a-openshift-service-ca\") pod \"maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n\" (UID: \"437f21e3-0aba-4826-bfcf-2ceffb0d127a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n"
Jul 30 01:16:49.984901 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:49.984761    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhmq7\" (UniqueName: \"kubernetes.io/projected/437f21e3-0aba-4826-bfcf-2ceffb0d127a-kube-api-access-bhmq7\") pod \"maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n\" (UID: \"437f21e3-0aba-4826-bfcf-2ceffb0d127a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n"
Jul 30 01:16:49.985458 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:49.985438    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/437f21e3-0aba-4826-bfcf-2ceffb0d127a-openshift-service-ca\") pod \"maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n\" (UID: \"437f21e3-0aba-4826-bfcf-2ceffb0d127a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n"
Jul 30 01:16:49.987206 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:49.987188    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/437f21e3-0aba-4826-bfcf-2ceffb0d127a-maas-api-tls\") pod \"maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n\" (UID: \"437f21e3-0aba-4826-bfcf-2ceffb0d127a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n"
Jul 30 01:16:49.993707 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:49.993679    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhmq7\" (UniqueName: \"kubernetes.io/projected/437f21e3-0aba-4826-bfcf-2ceffb0d127a-kube-api-access-bhmq7\") pod \"maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n\" (UID: \"437f21e3-0aba-4826-bfcf-2ceffb0d127a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n"
Jul 30 01:16:50.067258 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.067177    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n"
Jul 30 01:16:50.211417 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.211374    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n"]
Jul 30 01:16:50.213870 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:16:50.213840    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod437f21e3_0aba_4826_bfcf_2ceffb0d127a.slice/crio-c680611c35325222136c4c756247f00ec62328209467854809581f2cd240f7f8 WatchSource:0}: Error finding container c680611c35325222136c4c756247f00ec62328209467854809581f2cd240f7f8: Status 404 returned error can't find the container with id c680611c35325222136c4c756247f00ec62328209467854809581f2cd240f7f8
Jul 30 01:16:50.443277 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.443196    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj"]
Jul 30 01:16:50.447975 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.447957    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj"
Jul 30 01:16:50.450588 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.450562    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-095ac0a0-dockercfg-vpr9v\""
Jul 30 01:16:50.450718 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.450593    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-095ac0a0\""
Jul 30 01:16:50.456102 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.456077    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj"]
Jul 30 01:16:50.475152 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.475128    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df"]
Jul 30 01:16:50.478599 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.478584    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df"
Jul 30 01:16:50.487690 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.487658    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df"]
Jul 30 01:16:50.489588 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.489561    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d128028-36bf-4770-89ac-0a8db76df2e3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj\" (UID: \"7d128028-36bf-4770-89ac-0a8db76df2e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj"
Jul 30 01:16:50.489719 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.489639    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv27g\" (UniqueName: \"kubernetes.io/projected/7d128028-36bf-4770-89ac-0a8db76df2e3-kube-api-access-mv27g\") pod \"payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj\" (UID: \"7d128028-36bf-4770-89ac-0a8db76df2e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj"
Jul 30 01:16:50.590572 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.590513    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d128028-36bf-4770-89ac-0a8db76df2e3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj\" (UID: \"7d128028-36bf-4770-89ac-0a8db76df2e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj"
Jul 30 01:16:50.590748 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.590589    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73cb7108-62d3-4a0f-9fb0-65aa80fe1448-plugins-config-volume\") pod \"payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df\" (UID: \"73cb7108-62d3-4a0f-9fb0-65aa80fe1448\") " pod="openshift-ingress/payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df"
Jul 30 01:16:50.590748 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.590622    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv27g\" (UniqueName: \"kubernetes.io/projected/7d128028-36bf-4770-89ac-0a8db76df2e3-kube-api-access-mv27g\") pod \"payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj\" (UID: \"7d128028-36bf-4770-89ac-0a8db76df2e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj"
Jul 30 01:16:50.590748 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.590664    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pvnw\" (UniqueName: \"kubernetes.io/projected/73cb7108-62d3-4a0f-9fb0-65aa80fe1448-kube-api-access-9pvnw\") pod \"payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df\" (UID: \"73cb7108-62d3-4a0f-9fb0-65aa80fe1448\") " pod="openshift-ingress/payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df"
Jul 30 01:16:50.591598 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.591571    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d128028-36bf-4770-89ac-0a8db76df2e3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj\" (UID: \"7d128028-36bf-4770-89ac-0a8db76df2e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj"
Jul 30 01:16:50.599690 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.599669    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv27g\" (UniqueName: \"kubernetes.io/projected/7d128028-36bf-4770-89ac-0a8db76df2e3-kube-api-access-mv27g\") pod \"payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj\" (UID: \"7d128028-36bf-4770-89ac-0a8db76df2e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj"
Jul 30 01:16:50.691891 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.691848    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pvnw\" (UniqueName: \"kubernetes.io/projected/73cb7108-62d3-4a0f-9fb0-65aa80fe1448-kube-api-access-9pvnw\") pod \"payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df\" (UID: \"73cb7108-62d3-4a0f-9fb0-65aa80fe1448\") " pod="openshift-ingress/payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df"
Jul 30 01:16:50.692088 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.691966    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73cb7108-62d3-4a0f-9fb0-65aa80fe1448-plugins-config-volume\") pod \"payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df\" (UID: \"73cb7108-62d3-4a0f-9fb0-65aa80fe1448\") " pod="openshift-ingress/payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df"
Jul 30 01:16:50.692670 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.692648    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73cb7108-62d3-4a0f-9fb0-65aa80fe1448-plugins-config-volume\") pod \"payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df\" (UID: \"73cb7108-62d3-4a0f-9fb0-65aa80fe1448\") " pod="openshift-ingress/payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df"
Jul 30 01:16:50.699760 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.699700    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pvnw\" (UniqueName: \"kubernetes.io/projected/73cb7108-62d3-4a0f-9fb0-65aa80fe1448-kube-api-access-9pvnw\") pod \"payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df\" (UID: \"73cb7108-62d3-4a0f-9fb0-65aa80fe1448\") " pod="openshift-ingress/payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df"
Jul 30 01:16:50.760759 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.760732    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj"
Jul 30 01:16:50.790880 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.790834    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df"
Jul 30 01:16:50.918371 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:50.918312    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj"]
Jul 30 01:16:50.926103 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:16:50.926065    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d128028_36bf_4770_89ac_0a8db76df2e3.slice/crio-69775c5efa8fb4648e25fe933798504ba0abf3c2fbf93284ccc9ad99567b48c8 WatchSource:0}: Error finding container 69775c5efa8fb4648e25fe933798504ba0abf3c2fbf93284ccc9ad99567b48c8: Status 404 returned error can't find the container with id 69775c5efa8fb4648e25fe933798504ba0abf3c2fbf93284ccc9ad99567b48c8
Jul 30 01:16:51.160444 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:51.160406    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df"]
Jul 30 01:16:51.164838 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:16:51.164780    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73cb7108_62d3_4a0f_9fb0_65aa80fe1448.slice/crio-c29c85f453f72de9474fe011e70cf2b591401ef61b51708a7a9e97a197b7d3db WatchSource:0}: Error finding container c29c85f453f72de9474fe011e70cf2b591401ef61b51708a7a9e97a197b7d3db: Status 404 returned error can't find the container with id c29c85f453f72de9474fe011e70cf2b591401ef61b51708a7a9e97a197b7d3db
Jul 30 01:16:51.205690 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:51.205649    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df" event={"ID":"73cb7108-62d3-4a0f-9fb0-65aa80fe1448","Type":"ContainerStarted","Data":"c29c85f453f72de9474fe011e70cf2b591401ef61b51708a7a9e97a197b7d3db"}
Jul 30 01:16:51.207621 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:51.207592    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj" event={"ID":"7d128028-36bf-4770-89ac-0a8db76df2e3","Type":"ContainerStarted","Data":"8139bf1151a6caf7641b758d76217439fd27b047880ea11b50823e4274c83a51"}
Jul 30 01:16:51.207804 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:51.207787    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj" event={"ID":"7d128028-36bf-4770-89ac-0a8db76df2e3","Type":"ContainerStarted","Data":"69775c5efa8fb4648e25fe933798504ba0abf3c2fbf93284ccc9ad99567b48c8"}
Jul 30 01:16:51.207944 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:51.207926    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj"
Jul 30 01:16:51.209714 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:51.209683    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n" event={"ID":"437f21e3-0aba-4826-bfcf-2ceffb0d127a","Type":"ContainerStarted","Data":"e8c7162fcc82f9d45d080eef358c5aa5fe3082c4830d58b9d6a1fd435fbc26b7"}
Jul 30 01:16:51.209806 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:51.209734    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n" event={"ID":"437f21e3-0aba-4826-bfcf-2ceffb0d127a","Type":"ContainerStarted","Data":"c680611c35325222136c4c756247f00ec62328209467854809581f2cd240f7f8"}
Jul 30 01:16:51.209806 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:51.209776    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n"
Jul 30 01:16:51.225046 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:51.225004    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj" podStartSLOduration=1.2249868 podStartE2EDuration="1.2249868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:16:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:16:51.222958488 +0000 UTC m=+1998.445692307" watchObservedRunningTime="2026-07-30 01:16:51.2249868 +0000 UTC m=+1998.447720620"
Jul 30 01:16:51.240321 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:51.240275    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n" podStartSLOduration=1.561514625 podStartE2EDuration="2.240258771s" totalImagesPullingTime="678.744146ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:16:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:16:51.238800377 +0000 UTC m=+1998.461534236" watchObservedRunningTime="2026-07-30 01:16:51.240258771 +0000 UTC m=+1998.462992590"
Jul 30 01:16:52.216399 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:52.216355    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df" event={"ID":"73cb7108-62d3-4a0f-9fb0-65aa80fe1448","Type":"ContainerStarted","Data":"3149f3974a167f00bb8edcc1b7e5f958cca25fae9291f89882cbf71ab2006412"}
Jul 30 01:16:52.216831 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:52.216718    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df"
Jul 30 01:16:52.233772 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:52.233728    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df" podStartSLOduration=2.233714934 podStartE2EDuration="2.233714934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:16:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:16:52.233071182 +0000 UTC m=+1999.455805003" watchObservedRunningTime="2026-07-30 01:16:52.233714934 +0000 UTC m=+1999.456448777"
Jul 30 01:16:57.222553 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:16:57.222509    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n"
Jul 30 01:17:02.219049 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:02.219019    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj"
Jul 30 01:17:04.228142 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:04.228112    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df"
Jul 30 01:17:20.286811 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.286773    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n"]
Jul 30 01:17:20.287346 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.287086    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n" podUID="437f21e3-0aba-4826-bfcf-2ceffb0d127a" containerName="maas-api" containerID="cri-o://e8c7162fcc82f9d45d080eef358c5aa5fe3082c4830d58b9d6a1fd435fbc26b7" gracePeriod=30
Jul 30 01:17:20.339593 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.339562    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df"]
Jul 30 01:17:20.340046 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.340011    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df" podUID="73cb7108-62d3-4a0f-9fb0-65aa80fe1448" containerName="payload-processing" containerID="cri-o://3149f3974a167f00bb8edcc1b7e5f958cca25fae9291f89882cbf71ab2006412" gracePeriod=30
Jul 30 01:17:20.345914 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.345883    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj"]
Jul 30 01:17:20.346340 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.346285    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj" podUID="7d128028-36bf-4770-89ac-0a8db76df2e3" containerName="payload-pre-processing" containerID="cri-o://8139bf1151a6caf7641b758d76217439fd27b047880ea11b50823e4274c83a51" gracePeriod=30
Jul 30 01:17:20.612723 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.612692    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df"
Jul 30 01:17:20.661262 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.661239    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73cb7108-62d3-4a0f-9fb0-65aa80fe1448-plugins-config-volume\") pod \"73cb7108-62d3-4a0f-9fb0-65aa80fe1448\" (UID: \"73cb7108-62d3-4a0f-9fb0-65aa80fe1448\") "
Jul 30 01:17:20.661387 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.661319    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pvnw\" (UniqueName: \"kubernetes.io/projected/73cb7108-62d3-4a0f-9fb0-65aa80fe1448-kube-api-access-9pvnw\") pod \"73cb7108-62d3-4a0f-9fb0-65aa80fe1448\" (UID: \"73cb7108-62d3-4a0f-9fb0-65aa80fe1448\") "
Jul 30 01:17:20.661607 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.661589    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n"
Jul 30 01:17:20.661694 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.661655    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73cb7108-62d3-4a0f-9fb0-65aa80fe1448-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "73cb7108-62d3-4a0f-9fb0-65aa80fe1448" (UID: "73cb7108-62d3-4a0f-9fb0-65aa80fe1448"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:17:20.663471 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.663451    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73cb7108-62d3-4a0f-9fb0-65aa80fe1448-kube-api-access-9pvnw" (OuterVolumeSpecName: "kube-api-access-9pvnw") pod "73cb7108-62d3-4a0f-9fb0-65aa80fe1448" (UID: "73cb7108-62d3-4a0f-9fb0-65aa80fe1448"). InnerVolumeSpecName "kube-api-access-9pvnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:17:20.664954 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.664939    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj"
Jul 30 01:17:20.762380 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.762350    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/437f21e3-0aba-4826-bfcf-2ceffb0d127a-openshift-service-ca\") pod \"437f21e3-0aba-4826-bfcf-2ceffb0d127a\" (UID: \"437f21e3-0aba-4826-bfcf-2ceffb0d127a\") "
Jul 30 01:17:20.762578 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.762409    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d128028-36bf-4770-89ac-0a8db76df2e3-plugins-config-volume\") pod \"7d128028-36bf-4770-89ac-0a8db76df2e3\" (UID: \"7d128028-36bf-4770-89ac-0a8db76df2e3\") "
Jul 30 01:17:20.762578 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.762446    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhmq7\" (UniqueName: \"kubernetes.io/projected/437f21e3-0aba-4826-bfcf-2ceffb0d127a-kube-api-access-bhmq7\") pod \"437f21e3-0aba-4826-bfcf-2ceffb0d127a\" (UID: \"437f21e3-0aba-4826-bfcf-2ceffb0d127a\") "
Jul 30 01:17:20.762578 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.762464    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mv27g\" (UniqueName: \"kubernetes.io/projected/7d128028-36bf-4770-89ac-0a8db76df2e3-kube-api-access-mv27g\") pod \"7d128028-36bf-4770-89ac-0a8db76df2e3\" (UID: \"7d128028-36bf-4770-89ac-0a8db76df2e3\") "
Jul 30 01:17:20.762578 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.762484    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/437f21e3-0aba-4826-bfcf-2ceffb0d127a-maas-api-tls\") pod \"437f21e3-0aba-4826-bfcf-2ceffb0d127a\" (UID: \"437f21e3-0aba-4826-bfcf-2ceffb0d127a\") "
Jul 30 01:17:20.762822 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.762734    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/437f21e3-0aba-4826-bfcf-2ceffb0d127a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "437f21e3-0aba-4826-bfcf-2ceffb0d127a" (UID: "437f21e3-0aba-4826-bfcf-2ceffb0d127a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:17:20.762822 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.762746    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pvnw\" (UniqueName: \"kubernetes.io/projected/73cb7108-62d3-4a0f-9fb0-65aa80fe1448-kube-api-access-9pvnw\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:17:20.762822 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.762766    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73cb7108-62d3-4a0f-9fb0-65aa80fe1448-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:17:20.762957 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.762857    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d128028-36bf-4770-89ac-0a8db76df2e3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7d128028-36bf-4770-89ac-0a8db76df2e3" (UID: "7d128028-36bf-4770-89ac-0a8db76df2e3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:17:20.764619 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.764588    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/437f21e3-0aba-4826-bfcf-2ceffb0d127a-kube-api-access-bhmq7" (OuterVolumeSpecName: "kube-api-access-bhmq7") pod "437f21e3-0aba-4826-bfcf-2ceffb0d127a" (UID: "437f21e3-0aba-4826-bfcf-2ceffb0d127a"). InnerVolumeSpecName "kube-api-access-bhmq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:17:20.764711 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.764620    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d128028-36bf-4770-89ac-0a8db76df2e3-kube-api-access-mv27g" (OuterVolumeSpecName: "kube-api-access-mv27g") pod "7d128028-36bf-4770-89ac-0a8db76df2e3" (UID: "7d128028-36bf-4770-89ac-0a8db76df2e3"). InnerVolumeSpecName "kube-api-access-mv27g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:17:20.764711 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.764655    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/437f21e3-0aba-4826-bfcf-2ceffb0d127a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "437f21e3-0aba-4826-bfcf-2ceffb0d127a" (UID: "437f21e3-0aba-4826-bfcf-2ceffb0d127a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:17:20.863774 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.863693    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhmq7\" (UniqueName: \"kubernetes.io/projected/437f21e3-0aba-4826-bfcf-2ceffb0d127a-kube-api-access-bhmq7\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:17:20.863774 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.863720    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mv27g\" (UniqueName: \"kubernetes.io/projected/7d128028-36bf-4770-89ac-0a8db76df2e3-kube-api-access-mv27g\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:17:20.863774 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.863730    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/437f21e3-0aba-4826-bfcf-2ceffb0d127a-maas-api-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:17:20.863774 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.863739    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/437f21e3-0aba-4826-bfcf-2ceffb0d127a-openshift-service-ca\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:17:20.863774 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:20.863748    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d128028-36bf-4770-89ac-0a8db76df2e3-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:17:21.376274 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.376246    2546 generic.go:356] "Generic (PLEG): container finished" podID="73cb7108-62d3-4a0f-9fb0-65aa80fe1448" containerID="3149f3974a167f00bb8edcc1b7e5f958cca25fae9291f89882cbf71ab2006412" exitCode=0
Jul 30 01:17:21.376663 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.376288    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df"
Jul 30 01:17:21.376663 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.376325    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df" event={"ID":"73cb7108-62d3-4a0f-9fb0-65aa80fe1448","Type":"ContainerDied","Data":"3149f3974a167f00bb8edcc1b7e5f958cca25fae9291f89882cbf71ab2006412"}
Jul 30 01:17:21.376663 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.376376    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df" event={"ID":"73cb7108-62d3-4a0f-9fb0-65aa80fe1448","Type":"ContainerDied","Data":"c29c85f453f72de9474fe011e70cf2b591401ef61b51708a7a9e97a197b7d3db"}
Jul 30 01:17:21.376663 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.376400    2546 scope.go:117] "RemoveContainer" containerID="3149f3974a167f00bb8edcc1b7e5f958cca25fae9291f89882cbf71ab2006412"
Jul 30 01:17:21.377808 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.377660    2546 generic.go:356] "Generic (PLEG): container finished" podID="7d128028-36bf-4770-89ac-0a8db76df2e3" containerID="8139bf1151a6caf7641b758d76217439fd27b047880ea11b50823e4274c83a51" exitCode=0
Jul 30 01:17:21.377808 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.377704    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj"
Jul 30 01:17:21.377808 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.377747    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj" event={"ID":"7d128028-36bf-4770-89ac-0a8db76df2e3","Type":"ContainerDied","Data":"8139bf1151a6caf7641b758d76217439fd27b047880ea11b50823e4274c83a51"}
Jul 30 01:17:21.377808 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.377796    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj" event={"ID":"7d128028-36bf-4770-89ac-0a8db76df2e3","Type":"ContainerDied","Data":"69775c5efa8fb4648e25fe933798504ba0abf3c2fbf93284ccc9ad99567b48c8"}
Jul 30 01:17:21.379190 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.379167    2546 generic.go:356] "Generic (PLEG): container finished" podID="437f21e3-0aba-4826-bfcf-2ceffb0d127a" containerID="e8c7162fcc82f9d45d080eef358c5aa5fe3082c4830d58b9d6a1fd435fbc26b7" exitCode=0
Jul 30 01:17:21.379294 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.379211    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n"
Jul 30 01:17:21.379294 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.379215    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n" event={"ID":"437f21e3-0aba-4826-bfcf-2ceffb0d127a","Type":"ContainerDied","Data":"e8c7162fcc82f9d45d080eef358c5aa5fe3082c4830d58b9d6a1fd435fbc26b7"}
Jul 30 01:17:21.379294 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.379251    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n" event={"ID":"437f21e3-0aba-4826-bfcf-2ceffb0d127a","Type":"ContainerDied","Data":"c680611c35325222136c4c756247f00ec62328209467854809581f2cd240f7f8"}
Jul 30 01:17:21.385879 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.385863    2546 scope.go:117] "RemoveContainer" containerID="3149f3974a167f00bb8edcc1b7e5f958cca25fae9291f89882cbf71ab2006412"
Jul 30 01:17:21.386143 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:17:21.386127    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3149f3974a167f00bb8edcc1b7e5f958cca25fae9291f89882cbf71ab2006412\": container with ID starting with 3149f3974a167f00bb8edcc1b7e5f958cca25fae9291f89882cbf71ab2006412 not found: ID does not exist" containerID="3149f3974a167f00bb8edcc1b7e5f958cca25fae9291f89882cbf71ab2006412"
Jul 30 01:17:21.386222 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.386150    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3149f3974a167f00bb8edcc1b7e5f958cca25fae9291f89882cbf71ab2006412"} err="failed to get container status \"3149f3974a167f00bb8edcc1b7e5f958cca25fae9291f89882cbf71ab2006412\": rpc error: code = NotFound desc = could not find container \"3149f3974a167f00bb8edcc1b7e5f958cca25fae9291f89882cbf71ab2006412\": container with ID starting with 3149f3974a167f00bb8edcc1b7e5f958cca25fae9291f89882cbf71ab2006412 not found: ID does not exist"
Jul 30 01:17:21.386222 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.386165    2546 scope.go:117] "RemoveContainer" containerID="8139bf1151a6caf7641b758d76217439fd27b047880ea11b50823e4274c83a51"
Jul 30 01:17:21.393842 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.393821    2546 scope.go:117] "RemoveContainer" containerID="8139bf1151a6caf7641b758d76217439fd27b047880ea11b50823e4274c83a51"
Jul 30 01:17:21.394069 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:17:21.394052    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8139bf1151a6caf7641b758d76217439fd27b047880ea11b50823e4274c83a51\": container with ID starting with 8139bf1151a6caf7641b758d76217439fd27b047880ea11b50823e4274c83a51 not found: ID does not exist" containerID="8139bf1151a6caf7641b758d76217439fd27b047880ea11b50823e4274c83a51"
Jul 30 01:17:21.394121 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.394074    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8139bf1151a6caf7641b758d76217439fd27b047880ea11b50823e4274c83a51"} err="failed to get container status \"8139bf1151a6caf7641b758d76217439fd27b047880ea11b50823e4274c83a51\": rpc error: code = NotFound desc = could not find container \"8139bf1151a6caf7641b758d76217439fd27b047880ea11b50823e4274c83a51\": container with ID starting with 8139bf1151a6caf7641b758d76217439fd27b047880ea11b50823e4274c83a51 not found: ID does not exist"
Jul 30 01:17:21.394121 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.394088    2546 scope.go:117] "RemoveContainer" containerID="e8c7162fcc82f9d45d080eef358c5aa5fe3082c4830d58b9d6a1fd435fbc26b7"
Jul 30 01:17:21.401083 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.401063    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n"]
Jul 30 01:17:21.401873 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.401855    2546 scope.go:117] "RemoveContainer" containerID="e8c7162fcc82f9d45d080eef358c5aa5fe3082c4830d58b9d6a1fd435fbc26b7"
Jul 30 01:17:21.402107 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:17:21.402091    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8c7162fcc82f9d45d080eef358c5aa5fe3082c4830d58b9d6a1fd435fbc26b7\": container with ID starting with e8c7162fcc82f9d45d080eef358c5aa5fe3082c4830d58b9d6a1fd435fbc26b7 not found: ID does not exist" containerID="e8c7162fcc82f9d45d080eef358c5aa5fe3082c4830d58b9d6a1fd435fbc26b7"
Jul 30 01:17:21.402179 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.402113    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8c7162fcc82f9d45d080eef358c5aa5fe3082c4830d58b9d6a1fd435fbc26b7"} err="failed to get container status \"e8c7162fcc82f9d45d080eef358c5aa5fe3082c4830d58b9d6a1fd435fbc26b7\": rpc error: code = NotFound desc = could not find container \"e8c7162fcc82f9d45d080eef358c5aa5fe3082c4830d58b9d6a1fd435fbc26b7\": container with ID starting with e8c7162fcc82f9d45d080eef358c5aa5fe3082c4830d58b9d6a1fd435fbc26b7 not found: ID does not exist"
Jul 30 01:17:21.405155 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.405133    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-095ac0a0-59cb8f5c7f-jjc5n"]
Jul 30 01:17:21.415198 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.415178    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj"]
Jul 30 01:17:21.419215 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.419193    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-095ac0a0-7fb77bd644-xmgpj"]
Jul 30 01:17:21.428865 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.428842    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df"]
Jul 30 01:17:21.430821 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:21.430803    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-095ac0a0-867854c55d-2x5df"]
Jul 30 01:17:23.358273 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:23.358242    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="437f21e3-0aba-4826-bfcf-2ceffb0d127a" path="/var/lib/kubelet/pods/437f21e3-0aba-4826-bfcf-2ceffb0d127a/volumes"
Jul 30 01:17:23.358754 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:23.358587    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73cb7108-62d3-4a0f-9fb0-65aa80fe1448" path="/var/lib/kubelet/pods/73cb7108-62d3-4a0f-9fb0-65aa80fe1448/volumes"
Jul 30 01:17:23.358904 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:23.358891    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d128028-36bf-4770-89ac-0a8db76df2e3" path="/var/lib/kubelet/pods/7d128028-36bf-4770-89ac-0a8db76df2e3/volumes"
Jul 30 01:17:25.782878 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:25.782846    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh"]
Jul 30 01:17:25.783273 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:25.783259    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="437f21e3-0aba-4826-bfcf-2ceffb0d127a" containerName="maas-api"
Jul 30 01:17:25.783323 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:25.783275    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="437f21e3-0aba-4826-bfcf-2ceffb0d127a" containerName="maas-api"
Jul 30 01:17:25.783323 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:25.783288    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73cb7108-62d3-4a0f-9fb0-65aa80fe1448" containerName="payload-processing"
Jul 30 01:17:25.783323 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:25.783297    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="73cb7108-62d3-4a0f-9fb0-65aa80fe1448" containerName="payload-processing"
Jul 30 01:17:25.783323 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:25.783308    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d128028-36bf-4770-89ac-0a8db76df2e3" containerName="payload-pre-processing"
Jul 30 01:17:25.783323 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:25.783314    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d128028-36bf-4770-89ac-0a8db76df2e3" containerName="payload-pre-processing"
Jul 30 01:17:25.783477 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:25.783374    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d128028-36bf-4770-89ac-0a8db76df2e3" containerName="payload-pre-processing"
Jul 30 01:17:25.783477 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:25.783385    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="73cb7108-62d3-4a0f-9fb0-65aa80fe1448" containerName="payload-processing"
Jul 30 01:17:25.783477 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:25.783395    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="437f21e3-0aba-4826-bfcf-2ceffb0d127a" containerName="maas-api"
Jul 30 01:17:25.788356 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:25.788333    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh"
Jul 30 01:17:25.791160 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:25.790880    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-14be608d-serving-cert\""
Jul 30 01:17:25.798993 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:25.798965    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh"]
Jul 30 01:17:25.806802 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:25.806772    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzpvc\" (UniqueName: \"kubernetes.io/projected/cee23cff-efaf-4f3e-86e8-ebbf52dfd9de-kube-api-access-gzpvc\") pod \"maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh\" (UID: \"cee23cff-efaf-4f3e-86e8-ebbf52dfd9de\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh"
Jul 30 01:17:25.806912 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:25.806812    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cee23cff-efaf-4f3e-86e8-ebbf52dfd9de-maas-api-tls\") pod \"maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh\" (UID: \"cee23cff-efaf-4f3e-86e8-ebbf52dfd9de\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh"
Jul 30 01:17:25.806912 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:25.806839    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cee23cff-efaf-4f3e-86e8-ebbf52dfd9de-openshift-service-ca\") pod \"maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh\" (UID: \"cee23cff-efaf-4f3e-86e8-ebbf52dfd9de\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh"
Jul 30 01:17:25.908083 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:25.908048    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzpvc\" (UniqueName: \"kubernetes.io/projected/cee23cff-efaf-4f3e-86e8-ebbf52dfd9de-kube-api-access-gzpvc\") pod \"maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh\" (UID: \"cee23cff-efaf-4f3e-86e8-ebbf52dfd9de\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh"
Jul 30 01:17:25.908257 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:25.908089    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cee23cff-efaf-4f3e-86e8-ebbf52dfd9de-maas-api-tls\") pod \"maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh\" (UID: \"cee23cff-efaf-4f3e-86e8-ebbf52dfd9de\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh"
Jul 30 01:17:25.908257 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:25.908119    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cee23cff-efaf-4f3e-86e8-ebbf52dfd9de-openshift-service-ca\") pod \"maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh\" (UID: \"cee23cff-efaf-4f3e-86e8-ebbf52dfd9de\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh"
Jul 30 01:17:25.908369 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:17:25.908349    2546 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-serving-cert: secret "maas-api-e2e-derive-14be608d-serving-cert" not found
Jul 30 01:17:25.908426 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:17:25.908416    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cee23cff-efaf-4f3e-86e8-ebbf52dfd9de-maas-api-tls podName:cee23cff-efaf-4f3e-86e8-ebbf52dfd9de nodeName:}" failed. No retries permitted until 2026-07-30 01:17:26.408399322 +0000 UTC m=+2033.631133117 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cee23cff-efaf-4f3e-86e8-ebbf52dfd9de-maas-api-tls") pod "maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh" (UID: "cee23cff-efaf-4f3e-86e8-ebbf52dfd9de") : secret "maas-api-e2e-derive-14be608d-serving-cert" not found
Jul 30 01:17:25.908751 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:25.908732    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cee23cff-efaf-4f3e-86e8-ebbf52dfd9de-openshift-service-ca\") pod \"maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh\" (UID: \"cee23cff-efaf-4f3e-86e8-ebbf52dfd9de\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh"
Jul 30 01:17:25.918470 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:25.918439    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzpvc\" (UniqueName: \"kubernetes.io/projected/cee23cff-efaf-4f3e-86e8-ebbf52dfd9de-kube-api-access-gzpvc\") pod \"maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh\" (UID: \"cee23cff-efaf-4f3e-86e8-ebbf52dfd9de\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh"
Jul 30 01:17:26.411355 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.411321    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k"]
Jul 30 01:17:26.412698 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.412671    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cee23cff-efaf-4f3e-86e8-ebbf52dfd9de-maas-api-tls\") pod \"maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh\" (UID: \"cee23cff-efaf-4f3e-86e8-ebbf52dfd9de\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh"
Jul 30 01:17:26.415044 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.415024    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cee23cff-efaf-4f3e-86e8-ebbf52dfd9de-maas-api-tls\") pod \"maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh\" (UID: \"cee23cff-efaf-4f3e-86e8-ebbf52dfd9de\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh"
Jul 30 01:17:26.415253 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.415235    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k"
Jul 30 01:17:26.418115 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.418093    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-14be608d\""
Jul 30 01:17:26.418301 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.418289    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-14be608d-dockercfg-lwgtq\""
Jul 30 01:17:26.423412 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.423390    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k"]
Jul 30 01:17:26.425952 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.425932    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg"]
Jul 30 01:17:26.429684 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.429661    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg"
Jul 30 01:17:26.441195 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.441163    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg"]
Jul 30 01:17:26.513813 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.513779    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d4ea267-e8ad-4030-be9f-98c715707cf3-plugins-config-volume\") pod \"payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg\" (UID: \"6d4ea267-e8ad-4030-be9f-98c715707cf3\") " pod="openshift-ingress/payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg"
Jul 30 01:17:26.513966 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.513832    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blzpk\" (UniqueName: \"kubernetes.io/projected/e9ac8ba6-2afb-495e-9e6c-8ac5975109d5-kube-api-access-blzpk\") pod \"payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k\" (UID: \"e9ac8ba6-2afb-495e-9e6c-8ac5975109d5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k"
Jul 30 01:17:26.513966 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.513896    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbmkn\" (UniqueName: \"kubernetes.io/projected/6d4ea267-e8ad-4030-be9f-98c715707cf3-kube-api-access-cbmkn\") pod \"payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg\" (UID: \"6d4ea267-e8ad-4030-be9f-98c715707cf3\") " pod="openshift-ingress/payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg"
Jul 30 01:17:26.513966 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.513943    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9ac8ba6-2afb-495e-9e6c-8ac5975109d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k\" (UID: \"e9ac8ba6-2afb-495e-9e6c-8ac5975109d5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k"
Jul 30 01:17:26.615111 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.615076    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d4ea267-e8ad-4030-be9f-98c715707cf3-plugins-config-volume\") pod \"payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg\" (UID: \"6d4ea267-e8ad-4030-be9f-98c715707cf3\") " pod="openshift-ingress/payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg"
Jul 30 01:17:26.615305 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.615121    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blzpk\" (UniqueName: \"kubernetes.io/projected/e9ac8ba6-2afb-495e-9e6c-8ac5975109d5-kube-api-access-blzpk\") pod \"payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k\" (UID: \"e9ac8ba6-2afb-495e-9e6c-8ac5975109d5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k"
Jul 30 01:17:26.615305 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.615151    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbmkn\" (UniqueName: \"kubernetes.io/projected/6d4ea267-e8ad-4030-be9f-98c715707cf3-kube-api-access-cbmkn\") pod \"payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg\" (UID: \"6d4ea267-e8ad-4030-be9f-98c715707cf3\") " pod="openshift-ingress/payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg"
Jul 30 01:17:26.615305 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.615179    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9ac8ba6-2afb-495e-9e6c-8ac5975109d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k\" (UID: \"e9ac8ba6-2afb-495e-9e6c-8ac5975109d5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k"
Jul 30 01:17:26.615828 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.615807    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d4ea267-e8ad-4030-be9f-98c715707cf3-plugins-config-volume\") pod \"payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg\" (UID: \"6d4ea267-e8ad-4030-be9f-98c715707cf3\") " pod="openshift-ingress/payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg"
Jul 30 01:17:26.615906 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.615889    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9ac8ba6-2afb-495e-9e6c-8ac5975109d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k\" (UID: \"e9ac8ba6-2afb-495e-9e6c-8ac5975109d5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k"
Jul 30 01:17:26.624689 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.624665    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbmkn\" (UniqueName: \"kubernetes.io/projected/6d4ea267-e8ad-4030-be9f-98c715707cf3-kube-api-access-cbmkn\") pod \"payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg\" (UID: \"6d4ea267-e8ad-4030-be9f-98c715707cf3\") " pod="openshift-ingress/payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg"
Jul 30 01:17:26.624844 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.624824    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blzpk\" (UniqueName: \"kubernetes.io/projected/e9ac8ba6-2afb-495e-9e6c-8ac5975109d5-kube-api-access-blzpk\") pod \"payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k\" (UID: \"e9ac8ba6-2afb-495e-9e6c-8ac5975109d5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k"
Jul 30 01:17:26.702208 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.702118    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh"
Jul 30 01:17:26.734832 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.734803    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k"
Jul 30 01:17:26.743928 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.743898    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg"
Jul 30 01:17:26.871046 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.871008    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh"]
Jul 30 01:17:26.873413 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:17:26.873387    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcee23cff_efaf_4f3e_86e8_ebbf52dfd9de.slice/crio-d80d3f6055a5063e80bef6495b2272c60852497278a7302d4cd41f010e9288ab WatchSource:0}: Error finding container d80d3f6055a5063e80bef6495b2272c60852497278a7302d4cd41f010e9288ab: Status 404 returned error can't find the container with id d80d3f6055a5063e80bef6495b2272c60852497278a7302d4cd41f010e9288ab
Jul 30 01:17:26.920490 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.920409    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k"]
Jul 30 01:17:26.923111 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:17:26.923071    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9ac8ba6_2afb_495e_9e6c_8ac5975109d5.slice/crio-010521e6ec118f573c7781640e813f821698b4d201620e0baa98cd80d0066204 WatchSource:0}: Error finding container 010521e6ec118f573c7781640e813f821698b4d201620e0baa98cd80d0066204: Status 404 returned error can't find the container with id 010521e6ec118f573c7781640e813f821698b4d201620e0baa98cd80d0066204
Jul 30 01:17:26.941863 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:26.941835    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg"]
Jul 30 01:17:26.944055 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:17:26.944021    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d4ea267_e8ad_4030_be9f_98c715707cf3.slice/crio-4a8a76db40dea52a7c79c3e0e4df7c7a626b371588a325ec810fe88134f58a6a WatchSource:0}: Error finding container 4a8a76db40dea52a7c79c3e0e4df7c7a626b371588a325ec810fe88134f58a6a: Status 404 returned error can't find the container with id 4a8a76db40dea52a7c79c3e0e4df7c7a626b371588a325ec810fe88134f58a6a
Jul 30 01:17:27.414745 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:27.414642    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k" event={"ID":"e9ac8ba6-2afb-495e-9e6c-8ac5975109d5","Type":"ContainerStarted","Data":"6e174fb0a84c0c775eeb4f5ddc43c6db8c9a074f993263f1de902eb84c8bca4a"}
Jul 30 01:17:27.414745 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:27.414717    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k" event={"ID":"e9ac8ba6-2afb-495e-9e6c-8ac5975109d5","Type":"ContainerStarted","Data":"010521e6ec118f573c7781640e813f821698b4d201620e0baa98cd80d0066204"}
Jul 30 01:17:27.414941 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:27.414749    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k"
Jul 30 01:17:27.416404 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:27.416372    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg" event={"ID":"6d4ea267-e8ad-4030-be9f-98c715707cf3","Type":"ContainerStarted","Data":"6e4915c7e66c4f09458250b209b86120409666ae788549b92f49025e061e3d72"}
Jul 30 01:17:27.416507 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:27.416425    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg" event={"ID":"6d4ea267-e8ad-4030-be9f-98c715707cf3","Type":"ContainerStarted","Data":"4a8a76db40dea52a7c79c3e0e4df7c7a626b371588a325ec810fe88134f58a6a"}
Jul 30 01:17:27.416507 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:27.416452    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg"
Jul 30 01:17:27.418139 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:27.418117    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh" event={"ID":"cee23cff-efaf-4f3e-86e8-ebbf52dfd9de","Type":"ContainerStarted","Data":"d80d3f6055a5063e80bef6495b2272c60852497278a7302d4cd41f010e9288ab"}
Jul 30 01:17:27.433562 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:27.433499    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k" podStartSLOduration=1.433480888 podStartE2EDuration="1.433480888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:17:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:17:27.430827816 +0000 UTC m=+2034.653561634" watchObservedRunningTime="2026-07-30 01:17:27.433480888 +0000 UTC m=+2034.656214689"
Jul 30 01:17:27.446875 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:27.446836    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg" podStartSLOduration=1.446824451 podStartE2EDuration="1.446824451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:17:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:17:27.445691247 +0000 UTC m=+2034.668425069" watchObservedRunningTime="2026-07-30 01:17:27.446824451 +0000 UTC m=+2034.669558268"
Jul 30 01:17:28.425482 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:28.425427    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh" event={"ID":"cee23cff-efaf-4f3e-86e8-ebbf52dfd9de","Type":"ContainerStarted","Data":"d48273687a3f3837551f32a1e0d312f2faa81b34db82ad468e4910326637163f"}
Jul 30 01:17:28.425973 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:28.425882    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh"
Jul 30 01:17:28.443481 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:28.443435    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh" podStartSLOduration=2.970357955 podStartE2EDuration="3.443422108s" totalImagesPullingTime="473.064153ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:17:28.442171502 +0000 UTC m=+2035.664905321" watchObservedRunningTime="2026-07-30 01:17:28.443422108 +0000 UTC m=+2035.666155926"
Jul 30 01:17:35.442708 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:35.442673    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh"
Jul 30 01:17:38.427658 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:38.427623    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k"
Jul 30 01:17:38.428038 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:38.427783    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg"
Jul 30 01:17:56.463456 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.463143    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh"]
Jul 30 01:17:56.464155 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.464090    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh" podUID="cee23cff-efaf-4f3e-86e8-ebbf52dfd9de" containerName="maas-api" containerID="cri-o://d48273687a3f3837551f32a1e0d312f2faa81b34db82ad468e4910326637163f" gracePeriod=30
Jul 30 01:17:56.520544 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.520502    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg"]
Jul 30 01:17:56.520762 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.520715    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg" podUID="6d4ea267-e8ad-4030-be9f-98c715707cf3" containerName="payload-processing" containerID="cri-o://6e4915c7e66c4f09458250b209b86120409666ae788549b92f49025e061e3d72" gracePeriod=30
Jul 30 01:17:56.529561 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.529536    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k"]
Jul 30 01:17:56.529831 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.529754    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k" podUID="e9ac8ba6-2afb-495e-9e6c-8ac5975109d5" containerName="payload-pre-processing" containerID="cri-o://6e174fb0a84c0c775eeb4f5ddc43c6db8c9a074f993263f1de902eb84c8bca4a" gracePeriod=30
Jul 30 01:17:56.776627 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.776600    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg"
Jul 30 01:17:56.834983 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.834958    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh"
Jul 30 01:17:56.846789 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.846768    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k"
Jul 30 01:17:56.888452 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.888412    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbmkn\" (UniqueName: \"kubernetes.io/projected/6d4ea267-e8ad-4030-be9f-98c715707cf3-kube-api-access-cbmkn\") pod \"6d4ea267-e8ad-4030-be9f-98c715707cf3\" (UID: \"6d4ea267-e8ad-4030-be9f-98c715707cf3\") "
Jul 30 01:17:56.888687 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.888511    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d4ea267-e8ad-4030-be9f-98c715707cf3-plugins-config-volume\") pod \"6d4ea267-e8ad-4030-be9f-98c715707cf3\" (UID: \"6d4ea267-e8ad-4030-be9f-98c715707cf3\") "
Jul 30 01:17:56.889071 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.889045    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d4ea267-e8ad-4030-be9f-98c715707cf3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6d4ea267-e8ad-4030-be9f-98c715707cf3" (UID: "6d4ea267-e8ad-4030-be9f-98c715707cf3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:17:56.891074 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.891052    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d4ea267-e8ad-4030-be9f-98c715707cf3-kube-api-access-cbmkn" (OuterVolumeSpecName: "kube-api-access-cbmkn") pod "6d4ea267-e8ad-4030-be9f-98c715707cf3" (UID: "6d4ea267-e8ad-4030-be9f-98c715707cf3"). InnerVolumeSpecName "kube-api-access-cbmkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:17:56.989532 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.989499    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzpvc\" (UniqueName: \"kubernetes.io/projected/cee23cff-efaf-4f3e-86e8-ebbf52dfd9de-kube-api-access-gzpvc\") pod \"cee23cff-efaf-4f3e-86e8-ebbf52dfd9de\" (UID: \"cee23cff-efaf-4f3e-86e8-ebbf52dfd9de\") "
Jul 30 01:17:56.989706 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.989613    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9ac8ba6-2afb-495e-9e6c-8ac5975109d5-plugins-config-volume\") pod \"e9ac8ba6-2afb-495e-9e6c-8ac5975109d5\" (UID: \"e9ac8ba6-2afb-495e-9e6c-8ac5975109d5\") "
Jul 30 01:17:56.989706 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.989652    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blzpk\" (UniqueName: \"kubernetes.io/projected/e9ac8ba6-2afb-495e-9e6c-8ac5975109d5-kube-api-access-blzpk\") pod \"e9ac8ba6-2afb-495e-9e6c-8ac5975109d5\" (UID: \"e9ac8ba6-2afb-495e-9e6c-8ac5975109d5\") "
Jul 30 01:17:56.989831 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.989719    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cee23cff-efaf-4f3e-86e8-ebbf52dfd9de-openshift-service-ca\") pod \"cee23cff-efaf-4f3e-86e8-ebbf52dfd9de\" (UID: \"cee23cff-efaf-4f3e-86e8-ebbf52dfd9de\") "
Jul 30 01:17:56.989831 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.989751    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cee23cff-efaf-4f3e-86e8-ebbf52dfd9de-maas-api-tls\") pod \"cee23cff-efaf-4f3e-86e8-ebbf52dfd9de\" (UID: \"cee23cff-efaf-4f3e-86e8-ebbf52dfd9de\") "
Jul 30 01:17:56.989988 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.989972    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d4ea267-e8ad-4030-be9f-98c715707cf3-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:17:56.990044 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.989993    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbmkn\" (UniqueName: \"kubernetes.io/projected/6d4ea267-e8ad-4030-be9f-98c715707cf3-kube-api-access-cbmkn\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:17:56.990044 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.990011    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9ac8ba6-2afb-495e-9e6c-8ac5975109d5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e9ac8ba6-2afb-495e-9e6c-8ac5975109d5" (UID: "e9ac8ba6-2afb-495e-9e6c-8ac5975109d5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:17:56.990139 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.990118    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cee23cff-efaf-4f3e-86e8-ebbf52dfd9de-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cee23cff-efaf-4f3e-86e8-ebbf52dfd9de" (UID: "cee23cff-efaf-4f3e-86e8-ebbf52dfd9de"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:17:56.991798 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.991773    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cee23cff-efaf-4f3e-86e8-ebbf52dfd9de-kube-api-access-gzpvc" (OuterVolumeSpecName: "kube-api-access-gzpvc") pod "cee23cff-efaf-4f3e-86e8-ebbf52dfd9de" (UID: "cee23cff-efaf-4f3e-86e8-ebbf52dfd9de"). InnerVolumeSpecName "kube-api-access-gzpvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:17:56.991905 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.991823    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9ac8ba6-2afb-495e-9e6c-8ac5975109d5-kube-api-access-blzpk" (OuterVolumeSpecName: "kube-api-access-blzpk") pod "e9ac8ba6-2afb-495e-9e6c-8ac5975109d5" (UID: "e9ac8ba6-2afb-495e-9e6c-8ac5975109d5"). InnerVolumeSpecName "kube-api-access-blzpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:17:56.991905 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:56.991854    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cee23cff-efaf-4f3e-86e8-ebbf52dfd9de-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cee23cff-efaf-4f3e-86e8-ebbf52dfd9de" (UID: "cee23cff-efaf-4f3e-86e8-ebbf52dfd9de"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:17:57.091229 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.091142    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cee23cff-efaf-4f3e-86e8-ebbf52dfd9de-openshift-service-ca\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:17:57.091229 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.091177    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cee23cff-efaf-4f3e-86e8-ebbf52dfd9de-maas-api-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:17:57.091229 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.091194    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzpvc\" (UniqueName: \"kubernetes.io/projected/cee23cff-efaf-4f3e-86e8-ebbf52dfd9de-kube-api-access-gzpvc\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:17:57.091229 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.091209    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9ac8ba6-2afb-495e-9e6c-8ac5975109d5-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:17:57.091229 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.091222    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blzpk\" (UniqueName: \"kubernetes.io/projected/e9ac8ba6-2afb-495e-9e6c-8ac5975109d5-kube-api-access-blzpk\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:17:57.577726 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.577692    2546 generic.go:356] "Generic (PLEG): container finished" podID="6d4ea267-e8ad-4030-be9f-98c715707cf3" containerID="6e4915c7e66c4f09458250b209b86120409666ae788549b92f49025e061e3d72" exitCode=0
Jul 30 01:17:57.578161 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.577733    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg"
Jul 30 01:17:57.578161 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.577780    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg" event={"ID":"6d4ea267-e8ad-4030-be9f-98c715707cf3","Type":"ContainerDied","Data":"6e4915c7e66c4f09458250b209b86120409666ae788549b92f49025e061e3d72"}
Jul 30 01:17:57.578161 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.577835    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg" event={"ID":"6d4ea267-e8ad-4030-be9f-98c715707cf3","Type":"ContainerDied","Data":"4a8a76db40dea52a7c79c3e0e4df7c7a626b371588a325ec810fe88134f58a6a"}
Jul 30 01:17:57.578161 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.577858    2546 scope.go:117] "RemoveContainer" containerID="6e4915c7e66c4f09458250b209b86120409666ae788549b92f49025e061e3d72"
Jul 30 01:17:57.579222 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.579088    2546 generic.go:356] "Generic (PLEG): container finished" podID="cee23cff-efaf-4f3e-86e8-ebbf52dfd9de" containerID="d48273687a3f3837551f32a1e0d312f2faa81b34db82ad468e4910326637163f" exitCode=0
Jul 30 01:17:57.579222 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.579133    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh"
Jul 30 01:17:57.579222 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.579178    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh" event={"ID":"cee23cff-efaf-4f3e-86e8-ebbf52dfd9de","Type":"ContainerDied","Data":"d48273687a3f3837551f32a1e0d312f2faa81b34db82ad468e4910326637163f"}
Jul 30 01:17:57.579222 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.579219    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh" event={"ID":"cee23cff-efaf-4f3e-86e8-ebbf52dfd9de","Type":"ContainerDied","Data":"d80d3f6055a5063e80bef6495b2272c60852497278a7302d4cd41f010e9288ab"}
Jul 30 01:17:57.580627 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.580606    2546 generic.go:356] "Generic (PLEG): container finished" podID="e9ac8ba6-2afb-495e-9e6c-8ac5975109d5" containerID="6e174fb0a84c0c775eeb4f5ddc43c6db8c9a074f993263f1de902eb84c8bca4a" exitCode=0
Jul 30 01:17:57.580740 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.580651    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k"
Jul 30 01:17:57.580740 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.580650    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k" event={"ID":"e9ac8ba6-2afb-495e-9e6c-8ac5975109d5","Type":"ContainerDied","Data":"6e174fb0a84c0c775eeb4f5ddc43c6db8c9a074f993263f1de902eb84c8bca4a"}
Jul 30 01:17:57.580929 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.580765    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k" event={"ID":"e9ac8ba6-2afb-495e-9e6c-8ac5975109d5","Type":"ContainerDied","Data":"010521e6ec118f573c7781640e813f821698b4d201620e0baa98cd80d0066204"}
Jul 30 01:17:57.588428 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.588409    2546 scope.go:117] "RemoveContainer" containerID="6e4915c7e66c4f09458250b209b86120409666ae788549b92f49025e061e3d72"
Jul 30 01:17:57.588721 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:17:57.588696    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e4915c7e66c4f09458250b209b86120409666ae788549b92f49025e061e3d72\": container with ID starting with 6e4915c7e66c4f09458250b209b86120409666ae788549b92f49025e061e3d72 not found: ID does not exist" containerID="6e4915c7e66c4f09458250b209b86120409666ae788549b92f49025e061e3d72"
Jul 30 01:17:57.588793 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.588732    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e4915c7e66c4f09458250b209b86120409666ae788549b92f49025e061e3d72"} err="failed to get container status \"6e4915c7e66c4f09458250b209b86120409666ae788549b92f49025e061e3d72\": rpc error: code = NotFound desc = could not find container \"6e4915c7e66c4f09458250b209b86120409666ae788549b92f49025e061e3d72\": container with ID starting with 6e4915c7e66c4f09458250b209b86120409666ae788549b92f49025e061e3d72 not found: ID does not exist"
Jul 30 01:17:57.588793 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.588754    2546 scope.go:117] "RemoveContainer" containerID="d48273687a3f3837551f32a1e0d312f2faa81b34db82ad468e4910326637163f"
Jul 30 01:17:57.597301 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.597284    2546 scope.go:117] "RemoveContainer" containerID="d48273687a3f3837551f32a1e0d312f2faa81b34db82ad468e4910326637163f"
Jul 30 01:17:57.597542 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:17:57.597509    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d48273687a3f3837551f32a1e0d312f2faa81b34db82ad468e4910326637163f\": container with ID starting with d48273687a3f3837551f32a1e0d312f2faa81b34db82ad468e4910326637163f not found: ID does not exist" containerID="d48273687a3f3837551f32a1e0d312f2faa81b34db82ad468e4910326637163f"
Jul 30 01:17:57.597612 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.597549    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d48273687a3f3837551f32a1e0d312f2faa81b34db82ad468e4910326637163f"} err="failed to get container status \"d48273687a3f3837551f32a1e0d312f2faa81b34db82ad468e4910326637163f\": rpc error: code = NotFound desc = could not find container \"d48273687a3f3837551f32a1e0d312f2faa81b34db82ad468e4910326637163f\": container with ID starting with d48273687a3f3837551f32a1e0d312f2faa81b34db82ad468e4910326637163f not found: ID does not exist"
Jul 30 01:17:57.597612 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.597567    2546 scope.go:117] "RemoveContainer" containerID="6e174fb0a84c0c775eeb4f5ddc43c6db8c9a074f993263f1de902eb84c8bca4a"
Jul 30 01:17:57.600894 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.600871    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k"]
Jul 30 01:17:57.604812 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.604796    2546 scope.go:117] "RemoveContainer" containerID="6e174fb0a84c0c775eeb4f5ddc43c6db8c9a074f993263f1de902eb84c8bca4a"
Jul 30 01:17:57.605046 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:17:57.605031    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e174fb0a84c0c775eeb4f5ddc43c6db8c9a074f993263f1de902eb84c8bca4a\": container with ID starting with 6e174fb0a84c0c775eeb4f5ddc43c6db8c9a074f993263f1de902eb84c8bca4a not found: ID does not exist" containerID="6e174fb0a84c0c775eeb4f5ddc43c6db8c9a074f993263f1de902eb84c8bca4a"
Jul 30 01:17:57.605091 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.605051    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e174fb0a84c0c775eeb4f5ddc43c6db8c9a074f993263f1de902eb84c8bca4a"} err="failed to get container status \"6e174fb0a84c0c775eeb4f5ddc43c6db8c9a074f993263f1de902eb84c8bca4a\": rpc error: code = NotFound desc = could not find container \"6e174fb0a84c0c775eeb4f5ddc43c6db8c9a074f993263f1de902eb84c8bca4a\": container with ID starting with 6e174fb0a84c0c775eeb4f5ddc43c6db8c9a074f993263f1de902eb84c8bca4a not found: ID does not exist"
Jul 30 01:17:57.611035 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.611015    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-14be608d-c6599f464-bm74k"]
Jul 30 01:17:57.624604 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.624558    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh"]
Jul 30 01:17:57.633345 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.633322    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-14be608d-5f9d6985dc-xv4qh"]
Jul 30 01:17:57.644361 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.644335    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg"]
Jul 30 01:17:57.646278 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:57.646258    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-14be608d-5d8cb5b95d-7ltdg"]
Jul 30 01:17:59.358213 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:59.358178    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d4ea267-e8ad-4030-be9f-98c715707cf3" path="/var/lib/kubelet/pods/6d4ea267-e8ad-4030-be9f-98c715707cf3/volumes"
Jul 30 01:17:59.358772 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:59.358752    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cee23cff-efaf-4f3e-86e8-ebbf52dfd9de" path="/var/lib/kubelet/pods/cee23cff-efaf-4f3e-86e8-ebbf52dfd9de/volumes"
Jul 30 01:17:59.359133 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:17:59.359116    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9ac8ba6-2afb-495e-9e6c-8ac5975109d5" path="/var/lib/kubelet/pods/e9ac8ba6-2afb-495e-9e6c-8ac5975109d5/volumes"
Jul 30 01:18:33.350971 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:18:33.350937    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 01:18:33.352490 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:18:33.352467    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 01:20:07.444713 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:07.444681    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r"]
Jul 30 01:20:07.445200 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:07.445123    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cee23cff-efaf-4f3e-86e8-ebbf52dfd9de" containerName="maas-api"
Jul 30 01:20:07.445200 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:07.445139    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="cee23cff-efaf-4f3e-86e8-ebbf52dfd9de" containerName="maas-api"
Jul 30 01:20:07.445200 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:07.445156    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d4ea267-e8ad-4030-be9f-98c715707cf3" containerName="payload-processing"
Jul 30 01:20:07.445200 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:07.445161    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d4ea267-e8ad-4030-be9f-98c715707cf3" containerName="payload-processing"
Jul 30 01:20:07.445200 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:07.445174    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9ac8ba6-2afb-495e-9e6c-8ac5975109d5" containerName="payload-pre-processing"
Jul 30 01:20:07.445200 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:07.445180    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9ac8ba6-2afb-495e-9e6c-8ac5975109d5" containerName="payload-pre-processing"
Jul 30 01:20:07.445397 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:07.445239    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="cee23cff-efaf-4f3e-86e8-ebbf52dfd9de" containerName="maas-api"
Jul 30 01:20:07.445397 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:07.445252    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9ac8ba6-2afb-495e-9e6c-8ac5975109d5" containerName="payload-pre-processing"
Jul 30 01:20:07.445397 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:07.445259    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d4ea267-e8ad-4030-be9f-98c715707cf3" containerName="payload-processing"
Jul 30 01:20:07.448393 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:07.448376    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r"
Jul 30 01:20:07.450875 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:07.450848    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-14e4ae87-serving-cert\""
Jul 30 01:20:07.459909 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:07.459882    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r"]
Jul 30 01:20:07.523781 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:07.523747    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33fd7e16-129e-4e2c-9f38-735cab4821b1-openshift-service-ca\") pod \"maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r\" (UID: \"33fd7e16-129e-4e2c-9f38-735cab4821b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r"
Jul 30 01:20:07.523959 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:07.523805    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfwmj\" (UniqueName: \"kubernetes.io/projected/33fd7e16-129e-4e2c-9f38-735cab4821b1-kube-api-access-gfwmj\") pod \"maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r\" (UID: \"33fd7e16-129e-4e2c-9f38-735cab4821b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r"
Jul 30 01:20:07.523959 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:07.523863    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33fd7e16-129e-4e2c-9f38-735cab4821b1-maas-api-tls\") pod \"maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r\" (UID: \"33fd7e16-129e-4e2c-9f38-735cab4821b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r"
Jul 30 01:20:07.625356 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:07.625316    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33fd7e16-129e-4e2c-9f38-735cab4821b1-openshift-service-ca\") pod \"maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r\" (UID: \"33fd7e16-129e-4e2c-9f38-735cab4821b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r"
Jul 30 01:20:07.625585 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:07.625385    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfwmj\" (UniqueName: \"kubernetes.io/projected/33fd7e16-129e-4e2c-9f38-735cab4821b1-kube-api-access-gfwmj\") pod \"maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r\" (UID: \"33fd7e16-129e-4e2c-9f38-735cab4821b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r"
Jul 30 01:20:07.625585 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:07.625420    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33fd7e16-129e-4e2c-9f38-735cab4821b1-maas-api-tls\") pod \"maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r\" (UID: \"33fd7e16-129e-4e2c-9f38-735cab4821b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r"
Jul 30 01:20:07.626097 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:07.626064    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33fd7e16-129e-4e2c-9f38-735cab4821b1-openshift-service-ca\") pod \"maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r\" (UID: \"33fd7e16-129e-4e2c-9f38-735cab4821b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r"
Jul 30 01:20:07.627921 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:07.627899    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33fd7e16-129e-4e2c-9f38-735cab4821b1-maas-api-tls\") pod \"maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r\" (UID: \"33fd7e16-129e-4e2c-9f38-735cab4821b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r"
Jul 30 01:20:07.633627 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:07.633605    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfwmj\" (UniqueName: \"kubernetes.io/projected/33fd7e16-129e-4e2c-9f38-735cab4821b1-kube-api-access-gfwmj\") pod \"maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r\" (UID: \"33fd7e16-129e-4e2c-9f38-735cab4821b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r"
Jul 30 01:20:07.760175 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:07.760129    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r"
Jul 30 01:20:07.903932 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:07.903905    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r"]
Jul 30 01:20:07.907169 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:20:07.907137    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33fd7e16_129e_4e2c_9f38_735cab4821b1.slice/crio-1eca177a5b498aca7055c0a964d9649ebb4fedb5767d43890b4ba9bcb9549c26 WatchSource:0}: Error finding container 1eca177a5b498aca7055c0a964d9649ebb4fedb5767d43890b4ba9bcb9549c26: Status 404 returned error can't find the container with id 1eca177a5b498aca7055c0a964d9649ebb4fedb5767d43890b4ba9bcb9549c26
Jul 30 01:20:08.072224 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.072140    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42"]
Jul 30 01:20:08.076709 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.076693    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42"
Jul 30 01:20:08.081133 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.081103    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-14e4ae87\""
Jul 30 01:20:08.081332 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.081316    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-14e4ae87-dockercfg-xbq7f\""
Jul 30 01:20:08.091383 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.091352    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42"]
Jul 30 01:20:08.106735 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.106709    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t"]
Jul 30 01:20:08.110161 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.110145    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t"
Jul 30 01:20:08.120143 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.120118    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t"]
Jul 30 01:20:08.131328 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.131300    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsp9s\" (UniqueName: \"kubernetes.io/projected/8a41605d-c7df-4f5a-8b51-fa31fc10bccb-kube-api-access-tsp9s\") pod \"payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t\" (UID: \"8a41605d-c7df-4f5a-8b51-fa31fc10bccb\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t"
Jul 30 01:20:08.131463 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.131345    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxgwc\" (UniqueName: \"kubernetes.io/projected/4ff5ee2b-a166-46aa-9798-a091ef30d9c7-kube-api-access-cxgwc\") pod \"payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42\" (UID: \"4ff5ee2b-a166-46aa-9798-a091ef30d9c7\") " pod="openshift-ingress/payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42"
Jul 30 01:20:08.131463 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.131373    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a41605d-c7df-4f5a-8b51-fa31fc10bccb-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t\" (UID: \"8a41605d-c7df-4f5a-8b51-fa31fc10bccb\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t"
Jul 30 01:20:08.131463 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.131434    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ff5ee2b-a166-46aa-9798-a091ef30d9c7-plugins-config-volume\") pod \"payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42\" (UID: \"4ff5ee2b-a166-46aa-9798-a091ef30d9c7\") " pod="openshift-ingress/payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42"
Jul 30 01:20:08.232473 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.232434    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsp9s\" (UniqueName: \"kubernetes.io/projected/8a41605d-c7df-4f5a-8b51-fa31fc10bccb-kube-api-access-tsp9s\") pod \"payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t\" (UID: \"8a41605d-c7df-4f5a-8b51-fa31fc10bccb\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t"
Jul 30 01:20:08.232692 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.232486    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxgwc\" (UniqueName: \"kubernetes.io/projected/4ff5ee2b-a166-46aa-9798-a091ef30d9c7-kube-api-access-cxgwc\") pod \"payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42\" (UID: \"4ff5ee2b-a166-46aa-9798-a091ef30d9c7\") " pod="openshift-ingress/payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42"
Jul 30 01:20:08.232692 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.232511    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a41605d-c7df-4f5a-8b51-fa31fc10bccb-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t\" (UID: \"8a41605d-c7df-4f5a-8b51-fa31fc10bccb\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t"
Jul 30 01:20:08.232692 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.232563    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ff5ee2b-a166-46aa-9798-a091ef30d9c7-plugins-config-volume\") pod \"payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42\" (UID: \"4ff5ee2b-a166-46aa-9798-a091ef30d9c7\") " pod="openshift-ingress/payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42"
Jul 30 01:20:08.233260 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.233234    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a41605d-c7df-4f5a-8b51-fa31fc10bccb-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t\" (UID: \"8a41605d-c7df-4f5a-8b51-fa31fc10bccb\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t"
Jul 30 01:20:08.233345 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.233265    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ff5ee2b-a166-46aa-9798-a091ef30d9c7-plugins-config-volume\") pod \"payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42\" (UID: \"4ff5ee2b-a166-46aa-9798-a091ef30d9c7\") " pod="openshift-ingress/payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42"
Jul 30 01:20:08.241573 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.241548    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxgwc\" (UniqueName: \"kubernetes.io/projected/4ff5ee2b-a166-46aa-9798-a091ef30d9c7-kube-api-access-cxgwc\") pod \"payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42\" (UID: \"4ff5ee2b-a166-46aa-9798-a091ef30d9c7\") " pod="openshift-ingress/payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42"
Jul 30 01:20:08.241671 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.241581    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsp9s\" (UniqueName: \"kubernetes.io/projected/8a41605d-c7df-4f5a-8b51-fa31fc10bccb-kube-api-access-tsp9s\") pod \"payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t\" (UID: \"8a41605d-c7df-4f5a-8b51-fa31fc10bccb\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t"
Jul 30 01:20:08.249923 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.249895    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r" event={"ID":"33fd7e16-129e-4e2c-9f38-735cab4821b1","Type":"ContainerStarted","Data":"1eca177a5b498aca7055c0a964d9649ebb4fedb5767d43890b4ba9bcb9549c26"}
Jul 30 01:20:08.390226 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.390192    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42"
Jul 30 01:20:08.420467 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.420439    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t"
Jul 30 01:20:08.552542 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.552496    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42"]
Jul 30 01:20:08.555122 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:20:08.555084    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ff5ee2b_a166_46aa_9798_a091ef30d9c7.slice/crio-4fe509c800a21d0a0df0a070423f0c0e49fee7a9fc3556fd28ecdf2d48e56228 WatchSource:0}: Error finding container 4fe509c800a21d0a0df0a070423f0c0e49fee7a9fc3556fd28ecdf2d48e56228: Status 404 returned error can't find the container with id 4fe509c800a21d0a0df0a070423f0c0e49fee7a9fc3556fd28ecdf2d48e56228
Jul 30 01:20:08.791863 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:08.791839    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t"]
Jul 30 01:20:08.794359 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:20:08.794323    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a41605d_c7df_4f5a_8b51_fa31fc10bccb.slice/crio-648b5faefac74d8c6d16c78a3b36680f4a1d66cb7a429390297e2b70e3637f19 WatchSource:0}: Error finding container 648b5faefac74d8c6d16c78a3b36680f4a1d66cb7a429390297e2b70e3637f19: Status 404 returned error can't find the container with id 648b5faefac74d8c6d16c78a3b36680f4a1d66cb7a429390297e2b70e3637f19
Jul 30 01:20:09.256514 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:09.256475    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42" event={"ID":"4ff5ee2b-a166-46aa-9798-a091ef30d9c7","Type":"ContainerStarted","Data":"4d6ba80b62b30c63c8dba625248dda8704a3f5237ab451be2a96cee110a3bfe9"}
Jul 30 01:20:09.256736 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:09.256556    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42" event={"ID":"4ff5ee2b-a166-46aa-9798-a091ef30d9c7","Type":"ContainerStarted","Data":"4fe509c800a21d0a0df0a070423f0c0e49fee7a9fc3556fd28ecdf2d48e56228"}
Jul 30 01:20:09.256736 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:09.256687    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42"
Jul 30 01:20:09.257938 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:09.257915    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t" event={"ID":"8a41605d-c7df-4f5a-8b51-fa31fc10bccb","Type":"ContainerStarted","Data":"64796c97cc681d98094b7aee613b73993702ac059d244b49963d7a135e337ad1"}
Jul 30 01:20:09.258054 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:09.257954    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t" event={"ID":"8a41605d-c7df-4f5a-8b51-fa31fc10bccb","Type":"ContainerStarted","Data":"648b5faefac74d8c6d16c78a3b36680f4a1d66cb7a429390297e2b70e3637f19"}
Jul 30 01:20:09.258054 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:09.257991    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t"
Jul 30 01:20:09.259347 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:09.259330    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r" event={"ID":"33fd7e16-129e-4e2c-9f38-735cab4821b1","Type":"ContainerStarted","Data":"fccc17b24023bc96d98989a3d7a8b4f0979925c2614de6508cbdc86351d5e199"}
Jul 30 01:20:09.259439 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:09.259424    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r"
Jul 30 01:20:09.274793 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:09.274751    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42" podStartSLOduration=1.27473857 podStartE2EDuration="1.27473857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:20:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:20:09.273025238 +0000 UTC m=+2196.495759056" watchObservedRunningTime="2026-07-30 01:20:09.27473857 +0000 UTC m=+2196.497472387"
Jul 30 01:20:09.289342 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:09.289290    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r" podStartSLOduration=1.913887902 podStartE2EDuration="2.289272354s" totalImagesPullingTime="375.384452ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:20:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:20:09.288480498 +0000 UTC m=+2196.511214343" watchObservedRunningTime="2026-07-30 01:20:09.289272354 +0000 UTC m=+2196.512006175"
Jul 30 01:20:09.307574 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:09.307500    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t" podStartSLOduration=1.307483656 podStartE2EDuration="1.307483656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:20:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:20:09.304959552 +0000 UTC m=+2196.527693370" watchObservedRunningTime="2026-07-30 01:20:09.307483656 +0000 UTC m=+2196.530217480"
Jul 30 01:20:15.270736 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:15.270701    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r"
Jul 30 01:20:20.266279 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:20.266245    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42"
Jul 30 01:20:20.266737 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:20.266427    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t"
Jul 30 01:20:49.065435 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.065398    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r"]
Jul 30 01:20:49.067593 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.067531    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r" podUID="33fd7e16-129e-4e2c-9f38-735cab4821b1" containerName="maas-api" containerID="cri-o://fccc17b24023bc96d98989a3d7a8b4f0979925c2614de6508cbdc86351d5e199" gracePeriod=30
Jul 30 01:20:49.119483 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.119452    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42"]
Jul 30 01:20:49.119781 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.119717    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42" podUID="4ff5ee2b-a166-46aa-9798-a091ef30d9c7" containerName="payload-processing" containerID="cri-o://4d6ba80b62b30c63c8dba625248dda8704a3f5237ab451be2a96cee110a3bfe9" gracePeriod=30
Jul 30 01:20:49.125852 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.125827    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t"]
Jul 30 01:20:49.126329 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.126282    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t" podUID="8a41605d-c7df-4f5a-8b51-fa31fc10bccb" containerName="payload-pre-processing" containerID="cri-o://64796c97cc681d98094b7aee613b73993702ac059d244b49963d7a135e337ad1" gracePeriod=30
Jul 30 01:20:49.382678 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.382654    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r"
Jul 30 01:20:49.398622 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.398591    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33fd7e16-129e-4e2c-9f38-735cab4821b1-maas-api-tls\") pod \"33fd7e16-129e-4e2c-9f38-735cab4821b1\" (UID: \"33fd7e16-129e-4e2c-9f38-735cab4821b1\") "
Jul 30 01:20:49.398810 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.398689    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfwmj\" (UniqueName: \"kubernetes.io/projected/33fd7e16-129e-4e2c-9f38-735cab4821b1-kube-api-access-gfwmj\") pod \"33fd7e16-129e-4e2c-9f38-735cab4821b1\" (UID: \"33fd7e16-129e-4e2c-9f38-735cab4821b1\") "
Jul 30 01:20:49.398810 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.398733    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33fd7e16-129e-4e2c-9f38-735cab4821b1-openshift-service-ca\") pod \"33fd7e16-129e-4e2c-9f38-735cab4821b1\" (UID: \"33fd7e16-129e-4e2c-9f38-735cab4821b1\") "
Jul 30 01:20:49.399140 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.399113    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33fd7e16-129e-4e2c-9f38-735cab4821b1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "33fd7e16-129e-4e2c-9f38-735cab4821b1" (UID: "33fd7e16-129e-4e2c-9f38-735cab4821b1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:20:49.401826 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.401797    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33fd7e16-129e-4e2c-9f38-735cab4821b1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "33fd7e16-129e-4e2c-9f38-735cab4821b1" (UID: "33fd7e16-129e-4e2c-9f38-735cab4821b1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:20:49.401940 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.401861    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33fd7e16-129e-4e2c-9f38-735cab4821b1-kube-api-access-gfwmj" (OuterVolumeSpecName: "kube-api-access-gfwmj") pod "33fd7e16-129e-4e2c-9f38-735cab4821b1" (UID: "33fd7e16-129e-4e2c-9f38-735cab4821b1"). InnerVolumeSpecName "kube-api-access-gfwmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:20:49.425321 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.425298    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42"
Jul 30 01:20:49.431787 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.431768    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t"
Jul 30 01:20:49.469624 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.469589    2546 generic.go:356] "Generic (PLEG): container finished" podID="33fd7e16-129e-4e2c-9f38-735cab4821b1" containerID="fccc17b24023bc96d98989a3d7a8b4f0979925c2614de6508cbdc86351d5e199" exitCode=0
Jul 30 01:20:49.469781 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.469630    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r"
Jul 30 01:20:49.469781 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.469678    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r" event={"ID":"33fd7e16-129e-4e2c-9f38-735cab4821b1","Type":"ContainerDied","Data":"fccc17b24023bc96d98989a3d7a8b4f0979925c2614de6508cbdc86351d5e199"}
Jul 30 01:20:49.469781 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.469726    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r" event={"ID":"33fd7e16-129e-4e2c-9f38-735cab4821b1","Type":"ContainerDied","Data":"1eca177a5b498aca7055c0a964d9649ebb4fedb5767d43890b4ba9bcb9549c26"}
Jul 30 01:20:49.469781 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.469754    2546 scope.go:117] "RemoveContainer" containerID="fccc17b24023bc96d98989a3d7a8b4f0979925c2614de6508cbdc86351d5e199"
Jul 30 01:20:49.471169 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.471147    2546 generic.go:356] "Generic (PLEG): container finished" podID="4ff5ee2b-a166-46aa-9798-a091ef30d9c7" containerID="4d6ba80b62b30c63c8dba625248dda8704a3f5237ab451be2a96cee110a3bfe9" exitCode=0
Jul 30 01:20:49.471271 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.471215    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42" event={"ID":"4ff5ee2b-a166-46aa-9798-a091ef30d9c7","Type":"ContainerDied","Data":"4d6ba80b62b30c63c8dba625248dda8704a3f5237ab451be2a96cee110a3bfe9"}
Jul 30 01:20:49.471271 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.471266    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42" event={"ID":"4ff5ee2b-a166-46aa-9798-a091ef30d9c7","Type":"ContainerDied","Data":"4fe509c800a21d0a0df0a070423f0c0e49fee7a9fc3556fd28ecdf2d48e56228"}
Jul 30 01:20:49.471386 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.471269    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42"
Jul 30 01:20:49.472712 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.472688    2546 generic.go:356] "Generic (PLEG): container finished" podID="8a41605d-c7df-4f5a-8b51-fa31fc10bccb" containerID="64796c97cc681d98094b7aee613b73993702ac059d244b49963d7a135e337ad1" exitCode=0
Jul 30 01:20:49.472873 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.472736    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t"
Jul 30 01:20:49.472987 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.472730    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t" event={"ID":"8a41605d-c7df-4f5a-8b51-fa31fc10bccb","Type":"ContainerDied","Data":"64796c97cc681d98094b7aee613b73993702ac059d244b49963d7a135e337ad1"}
Jul 30 01:20:49.473056 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.473008    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t" event={"ID":"8a41605d-c7df-4f5a-8b51-fa31fc10bccb","Type":"ContainerDied","Data":"648b5faefac74d8c6d16c78a3b36680f4a1d66cb7a429390297e2b70e3637f19"}
Jul 30 01:20:49.479618 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.479603    2546 scope.go:117] "RemoveContainer" containerID="fccc17b24023bc96d98989a3d7a8b4f0979925c2614de6508cbdc86351d5e199"
Jul 30 01:20:49.479857 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:20:49.479838    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fccc17b24023bc96d98989a3d7a8b4f0979925c2614de6508cbdc86351d5e199\": container with ID starting with fccc17b24023bc96d98989a3d7a8b4f0979925c2614de6508cbdc86351d5e199 not found: ID does not exist" containerID="fccc17b24023bc96d98989a3d7a8b4f0979925c2614de6508cbdc86351d5e199"
Jul 30 01:20:49.479946 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.479866    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fccc17b24023bc96d98989a3d7a8b4f0979925c2614de6508cbdc86351d5e199"} err="failed to get container status \"fccc17b24023bc96d98989a3d7a8b4f0979925c2614de6508cbdc86351d5e199\": rpc error: code = NotFound desc = could not find container \"fccc17b24023bc96d98989a3d7a8b4f0979925c2614de6508cbdc86351d5e199\": container with ID starting with fccc17b24023bc96d98989a3d7a8b4f0979925c2614de6508cbdc86351d5e199 not found: ID does not exist"
Jul 30 01:20:49.479946 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.479887    2546 scope.go:117] "RemoveContainer" containerID="4d6ba80b62b30c63c8dba625248dda8704a3f5237ab451be2a96cee110a3bfe9"
Jul 30 01:20:49.488492 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.488476    2546 scope.go:117] "RemoveContainer" containerID="4d6ba80b62b30c63c8dba625248dda8704a3f5237ab451be2a96cee110a3bfe9"
Jul 30 01:20:49.488808 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:20:49.488786    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d6ba80b62b30c63c8dba625248dda8704a3f5237ab451be2a96cee110a3bfe9\": container with ID starting with 4d6ba80b62b30c63c8dba625248dda8704a3f5237ab451be2a96cee110a3bfe9 not found: ID does not exist" containerID="4d6ba80b62b30c63c8dba625248dda8704a3f5237ab451be2a96cee110a3bfe9"
Jul 30 01:20:49.488908 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.488817    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d6ba80b62b30c63c8dba625248dda8704a3f5237ab451be2a96cee110a3bfe9"} err="failed to get container status \"4d6ba80b62b30c63c8dba625248dda8704a3f5237ab451be2a96cee110a3bfe9\": rpc error: code = NotFound desc = could not find container \"4d6ba80b62b30c63c8dba625248dda8704a3f5237ab451be2a96cee110a3bfe9\": container with ID starting with 4d6ba80b62b30c63c8dba625248dda8704a3f5237ab451be2a96cee110a3bfe9 not found: ID does not exist"
Jul 30 01:20:49.488908 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.488838    2546 scope.go:117] "RemoveContainer" containerID="64796c97cc681d98094b7aee613b73993702ac059d244b49963d7a135e337ad1"
Jul 30 01:20:49.494049 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.494023    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r"]
Jul 30 01:20:49.497282 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.497257    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-14e4ae87-6cb77577fb-bwk5r"]
Jul 30 01:20:49.499181 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.499162    2546 scope.go:117] "RemoveContainer" containerID="64796c97cc681d98094b7aee613b73993702ac059d244b49963d7a135e337ad1"
Jul 30 01:20:49.499371 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.499354    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ff5ee2b-a166-46aa-9798-a091ef30d9c7-plugins-config-volume\") pod \"4ff5ee2b-a166-46aa-9798-a091ef30d9c7\" (UID: \"4ff5ee2b-a166-46aa-9798-a091ef30d9c7\") "
Jul 30 01:20:49.499441 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.499387    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsp9s\" (UniqueName: \"kubernetes.io/projected/8a41605d-c7df-4f5a-8b51-fa31fc10bccb-kube-api-access-tsp9s\") pod \"8a41605d-c7df-4f5a-8b51-fa31fc10bccb\" (UID: \"8a41605d-c7df-4f5a-8b51-fa31fc10bccb\") "
Jul 30 01:20:49.499480 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:20:49.499441    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64796c97cc681d98094b7aee613b73993702ac059d244b49963d7a135e337ad1\": container with ID starting with 64796c97cc681d98094b7aee613b73993702ac059d244b49963d7a135e337ad1 not found: ID does not exist" containerID="64796c97cc681d98094b7aee613b73993702ac059d244b49963d7a135e337ad1"
Jul 30 01:20:49.499560 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.499468    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64796c97cc681d98094b7aee613b73993702ac059d244b49963d7a135e337ad1"} err="failed to get container status \"64796c97cc681d98094b7aee613b73993702ac059d244b49963d7a135e337ad1\": rpc error: code = NotFound desc = could not find container \"64796c97cc681d98094b7aee613b73993702ac059d244b49963d7a135e337ad1\": container with ID starting with 64796c97cc681d98094b7aee613b73993702ac059d244b49963d7a135e337ad1 not found: ID does not exist"
Jul 30 01:20:49.499560 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.499513    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxgwc\" (UniqueName: \"kubernetes.io/projected/4ff5ee2b-a166-46aa-9798-a091ef30d9c7-kube-api-access-cxgwc\") pod \"4ff5ee2b-a166-46aa-9798-a091ef30d9c7\" (UID: \"4ff5ee2b-a166-46aa-9798-a091ef30d9c7\") "
Jul 30 01:20:49.499685 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.499574    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a41605d-c7df-4f5a-8b51-fa31fc10bccb-plugins-config-volume\") pod \"8a41605d-c7df-4f5a-8b51-fa31fc10bccb\" (UID: \"8a41605d-c7df-4f5a-8b51-fa31fc10bccb\") "
Jul 30 01:20:49.499845 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.499826    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33fd7e16-129e-4e2c-9f38-735cab4821b1-maas-api-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:20:49.499845 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.499789    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ff5ee2b-a166-46aa-9798-a091ef30d9c7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4ff5ee2b-a166-46aa-9798-a091ef30d9c7" (UID: "4ff5ee2b-a166-46aa-9798-a091ef30d9c7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:20:49.499977 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.499853    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfwmj\" (UniqueName: \"kubernetes.io/projected/33fd7e16-129e-4e2c-9f38-735cab4821b1-kube-api-access-gfwmj\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:20:49.499977 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.499871    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33fd7e16-129e-4e2c-9f38-735cab4821b1-openshift-service-ca\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:20:49.499977 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.499972    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a41605d-c7df-4f5a-8b51-fa31fc10bccb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8a41605d-c7df-4f5a-8b51-fa31fc10bccb" (UID: "8a41605d-c7df-4f5a-8b51-fa31fc10bccb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:20:49.501511 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.501485    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ff5ee2b-a166-46aa-9798-a091ef30d9c7-kube-api-access-cxgwc" (OuterVolumeSpecName: "kube-api-access-cxgwc") pod "4ff5ee2b-a166-46aa-9798-a091ef30d9c7" (UID: "4ff5ee2b-a166-46aa-9798-a091ef30d9c7"). InnerVolumeSpecName "kube-api-access-cxgwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:20:49.501626 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.501569    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a41605d-c7df-4f5a-8b51-fa31fc10bccb-kube-api-access-tsp9s" (OuterVolumeSpecName: "kube-api-access-tsp9s") pod "8a41605d-c7df-4f5a-8b51-fa31fc10bccb" (UID: "8a41605d-c7df-4f5a-8b51-fa31fc10bccb"). InnerVolumeSpecName "kube-api-access-tsp9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:20:49.601125 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.601051    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxgwc\" (UniqueName: \"kubernetes.io/projected/4ff5ee2b-a166-46aa-9798-a091ef30d9c7-kube-api-access-cxgwc\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:20:49.601125 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.601079    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a41605d-c7df-4f5a-8b51-fa31fc10bccb-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:20:49.601125 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.601089    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ff5ee2b-a166-46aa-9798-a091ef30d9c7-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:20:49.601125 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.601097    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsp9s\" (UniqueName: \"kubernetes.io/projected/8a41605d-c7df-4f5a-8b51-fa31fc10bccb-kube-api-access-tsp9s\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:20:49.796826 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.796795    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t"]
Jul 30 01:20:49.801675 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.801652    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-14e4ae87-5986fc6f44-k252t"]
Jul 30 01:20:49.812109 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.812087    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42"]
Jul 30 01:20:49.815375 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:49.815351    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-14e4ae87-7d9b795fc8-m8h42"]
Jul 30 01:20:51.357762 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:51.357730    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33fd7e16-129e-4e2c-9f38-735cab4821b1" path="/var/lib/kubelet/pods/33fd7e16-129e-4e2c-9f38-735cab4821b1/volumes"
Jul 30 01:20:51.358150 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:51.358085    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ff5ee2b-a166-46aa-9798-a091ef30d9c7" path="/var/lib/kubelet/pods/4ff5ee2b-a166-46aa-9798-a091ef30d9c7/volumes"
Jul 30 01:20:51.358389 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:20:51.358376    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a41605d-c7df-4f5a-8b51-fa31fc10bccb" path="/var/lib/kubelet/pods/8a41605d-c7df-4f5a-8b51-fa31fc10bccb/volumes"
Jul 30 01:21:36.528898 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:21:36.528861    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ftprg"]
Jul 30 01:21:36.529405 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:21:36.529059    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ftprg" podUID="f39e270c-2047-478a-89bf-86e7b630271f" containerName="manager" containerID="cri-o://8a41350cac34babc224e94e37d7e671b63c187f56bc5fd00d00efe77852ece17" gracePeriod=10
Jul 30 01:21:37.714420 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:21:37.714391    2546 generic.go:356] "Generic (PLEG): container finished" podID="f39e270c-2047-478a-89bf-86e7b630271f" containerID="8a41350cac34babc224e94e37d7e671b63c187f56bc5fd00d00efe77852ece17" exitCode=0
Jul 30 01:21:37.714766 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:21:37.714462    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ftprg" event={"ID":"f39e270c-2047-478a-89bf-86e7b630271f","Type":"ContainerDied","Data":"8a41350cac34babc224e94e37d7e671b63c187f56bc5fd00d00efe77852ece17"}
Jul 30 01:21:38.299691 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:21:38.299669    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ftprg"
Jul 30 01:21:38.326300 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:21:38.326274    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9x4n\" (UniqueName: \"kubernetes.io/projected/f39e270c-2047-478a-89bf-86e7b630271f-kube-api-access-x9x4n\") pod \"f39e270c-2047-478a-89bf-86e7b630271f\" (UID: \"f39e270c-2047-478a-89bf-86e7b630271f\") "
Jul 30 01:21:38.326455 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:21:38.326410    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f39e270c-2047-478a-89bf-86e7b630271f-extensions-socket-volume\") pod \"f39e270c-2047-478a-89bf-86e7b630271f\" (UID: \"f39e270c-2047-478a-89bf-86e7b630271f\") "
Jul 30 01:21:38.326855 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:21:38.326827    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f39e270c-2047-478a-89bf-86e7b630271f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f39e270c-2047-478a-89bf-86e7b630271f" (UID: "f39e270c-2047-478a-89bf-86e7b630271f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 01:21:38.328581 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:21:38.328551    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f39e270c-2047-478a-89bf-86e7b630271f-kube-api-access-x9x4n" (OuterVolumeSpecName: "kube-api-access-x9x4n") pod "f39e270c-2047-478a-89bf-86e7b630271f" (UID: "f39e270c-2047-478a-89bf-86e7b630271f"). InnerVolumeSpecName "kube-api-access-x9x4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:21:38.427659 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:21:38.427584    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9x4n\" (UniqueName: \"kubernetes.io/projected/f39e270c-2047-478a-89bf-86e7b630271f-kube-api-access-x9x4n\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:21:38.427659 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:21:38.427610    2546 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f39e270c-2047-478a-89bf-86e7b630271f-extensions-socket-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:21:38.719955 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:21:38.719929    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ftprg"
Jul 30 01:21:38.720356 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:21:38.719925    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ftprg" event={"ID":"f39e270c-2047-478a-89bf-86e7b630271f","Type":"ContainerDied","Data":"3c8fb8220a2b21c24e3aba190c766ce86e8da9102bac4c5d710b4666aacb1740"}
Jul 30 01:21:38.720356 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:21:38.720066    2546 scope.go:117] "RemoveContainer" containerID="8a41350cac34babc224e94e37d7e671b63c187f56bc5fd00d00efe77852ece17"
Jul 30 01:21:38.742844 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:21:38.742822    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ftprg"]
Jul 30 01:21:38.750191 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:21:38.750167    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ftprg"]
Jul 30 01:21:39.358892 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:21:39.358861    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f39e270c-2047-478a-89bf-86e7b630271f" path="/var/lib/kubelet/pods/f39e270c-2047-478a-89bf-86e7b630271f/volumes"
Jul 30 01:22:52.150586 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:52.150550    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfwrv"]
Jul 30 01:22:52.151105 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:52.150937    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a41605d-c7df-4f5a-8b51-fa31fc10bccb" containerName="payload-pre-processing"
Jul 30 01:22:52.151105 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:52.150949    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a41605d-c7df-4f5a-8b51-fa31fc10bccb" containerName="payload-pre-processing"
Jul 30 01:22:52.151105 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:52.150961    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f39e270c-2047-478a-89bf-86e7b630271f" containerName="manager"
Jul 30 01:22:52.151105 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:52.150967    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="f39e270c-2047-478a-89bf-86e7b630271f" containerName="manager"
Jul 30 01:22:52.151105 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:52.150973    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33fd7e16-129e-4e2c-9f38-735cab4821b1" containerName="maas-api"
Jul 30 01:22:52.151105 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:52.150979    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="33fd7e16-129e-4e2c-9f38-735cab4821b1" containerName="maas-api"
Jul 30 01:22:52.151105 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:52.150986    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ff5ee2b-a166-46aa-9798-a091ef30d9c7" containerName="payload-processing"
Jul 30 01:22:52.151105 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:52.150991    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ff5ee2b-a166-46aa-9798-a091ef30d9c7" containerName="payload-processing"
Jul 30 01:22:52.151105 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:52.151048    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a41605d-c7df-4f5a-8b51-fa31fc10bccb" containerName="payload-pre-processing"
Jul 30 01:22:52.151105 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:52.151056    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ff5ee2b-a166-46aa-9798-a091ef30d9c7" containerName="payload-processing"
Jul 30 01:22:52.151105 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:52.151063    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="33fd7e16-129e-4e2c-9f38-735cab4821b1" containerName="maas-api"
Jul 30 01:22:52.151105 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:52.151069    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="f39e270c-2047-478a-89bf-86e7b630271f" containerName="manager"
Jul 30 01:22:52.154120 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:52.154103    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfwrv"
Jul 30 01:22:52.159387 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:52.159364    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dgwdc\""
Jul 30 01:22:52.171850 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:52.171821    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfwrv"]
Jul 30 01:22:52.255079 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:52.255047    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/147f1e75-25ab-45d1-8e8c-475c4327194f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bfwrv\" (UID: \"147f1e75-25ab-45d1-8e8c-475c4327194f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfwrv"
Jul 30 01:22:52.255249 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:52.255090    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl2xx\" (UniqueName: \"kubernetes.io/projected/147f1e75-25ab-45d1-8e8c-475c4327194f-kube-api-access-tl2xx\") pod \"kuadrant-operator-controller-manager-86878c769f-bfwrv\" (UID: \"147f1e75-25ab-45d1-8e8c-475c4327194f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfwrv"
Jul 30 01:22:52.356404 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:52.356372    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/147f1e75-25ab-45d1-8e8c-475c4327194f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bfwrv\" (UID: \"147f1e75-25ab-45d1-8e8c-475c4327194f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfwrv"
Jul 30 01:22:52.356606 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:52.356413    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl2xx\" (UniqueName: \"kubernetes.io/projected/147f1e75-25ab-45d1-8e8c-475c4327194f-kube-api-access-tl2xx\") pod \"kuadrant-operator-controller-manager-86878c769f-bfwrv\" (UID: \"147f1e75-25ab-45d1-8e8c-475c4327194f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfwrv"
Jul 30 01:22:52.356793 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:52.356771    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/147f1e75-25ab-45d1-8e8c-475c4327194f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bfwrv\" (UID: \"147f1e75-25ab-45d1-8e8c-475c4327194f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfwrv"
Jul 30 01:22:52.366847 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:52.366825    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl2xx\" (UniqueName: \"kubernetes.io/projected/147f1e75-25ab-45d1-8e8c-475c4327194f-kube-api-access-tl2xx\") pod \"kuadrant-operator-controller-manager-86878c769f-bfwrv\" (UID: \"147f1e75-25ab-45d1-8e8c-475c4327194f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfwrv"
Jul 30 01:22:52.464602 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:52.464509    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfwrv"
Jul 30 01:22:52.607662 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:52.607632    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfwrv"]
Jul 30 01:22:52.611173 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:22:52.611145    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod147f1e75_25ab_45d1_8e8c_475c4327194f.slice/crio-8c1394b854e18c1fb75c18b7e35c18f31eb431e9612976105c1009e120685a09 WatchSource:0}: Error finding container 8c1394b854e18c1fb75c18b7e35c18f31eb431e9612976105c1009e120685a09: Status 404 returned error can't find the container with id 8c1394b854e18c1fb75c18b7e35c18f31eb431e9612976105c1009e120685a09
Jul 30 01:22:53.081426 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:53.081386    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfwrv" event={"ID":"147f1e75-25ab-45d1-8e8c-475c4327194f","Type":"ContainerStarted","Data":"a379f5a68cbab56d022fd50cffdc35fc53a60753bc8472f3a8b04bec692a4a9d"}
Jul 30 01:22:53.081640 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:53.081449    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfwrv"
Jul 30 01:22:53.081640 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:53.081472    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfwrv" event={"ID":"147f1e75-25ab-45d1-8e8c-475c4327194f","Type":"ContainerStarted","Data":"8c1394b854e18c1fb75c18b7e35c18f31eb431e9612976105c1009e120685a09"}
Jul 30 01:22:53.101732 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:22:53.101685    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfwrv" podStartSLOduration=1.101670188 podStartE2EDuration="1.101670188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:22:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:22:53.099946629 +0000 UTC m=+2360.322680448" watchObservedRunningTime="2026-07-30 01:22:53.101670188 +0000 UTC m=+2360.324404060"
Jul 30 01:23:04.088315 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:04.088282    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfwrv"
Jul 30 01:23:15.042818 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.042785    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd"]
Jul 30 01:23:15.046200 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.046177    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd"
Jul 30 01:23:15.049065 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.049042    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-671ba06b-serving-cert\""
Jul 30 01:23:15.056406 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.056381    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd"]
Jul 30 01:23:15.165762 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.165727    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a404f130-bf63-4f08-9b60-8afefbe9a561-openshift-service-ca\") pod \"maas-api-e2e-mt-671ba06b-585685dcd-5wbnd\" (UID: \"a404f130-bf63-4f08-9b60-8afefbe9a561\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd"
Jul 30 01:23:15.165942 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.165775    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnrnm\" (UniqueName: \"kubernetes.io/projected/a404f130-bf63-4f08-9b60-8afefbe9a561-kube-api-access-cnrnm\") pod \"maas-api-e2e-mt-671ba06b-585685dcd-5wbnd\" (UID: \"a404f130-bf63-4f08-9b60-8afefbe9a561\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd"
Jul 30 01:23:15.165942 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.165887    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a404f130-bf63-4f08-9b60-8afefbe9a561-maas-api-tls\") pod \"maas-api-e2e-mt-671ba06b-585685dcd-5wbnd\" (UID: \"a404f130-bf63-4f08-9b60-8afefbe9a561\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd"
Jul 30 01:23:15.266670 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.266624    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a404f130-bf63-4f08-9b60-8afefbe9a561-maas-api-tls\") pod \"maas-api-e2e-mt-671ba06b-585685dcd-5wbnd\" (UID: \"a404f130-bf63-4f08-9b60-8afefbe9a561\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd"
Jul 30 01:23:15.266847 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.266702    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a404f130-bf63-4f08-9b60-8afefbe9a561-openshift-service-ca\") pod \"maas-api-e2e-mt-671ba06b-585685dcd-5wbnd\" (UID: \"a404f130-bf63-4f08-9b60-8afefbe9a561\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd"
Jul 30 01:23:15.266847 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.266743    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnrnm\" (UniqueName: \"kubernetes.io/projected/a404f130-bf63-4f08-9b60-8afefbe9a561-kube-api-access-cnrnm\") pod \"maas-api-e2e-mt-671ba06b-585685dcd-5wbnd\" (UID: \"a404f130-bf63-4f08-9b60-8afefbe9a561\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd"
Jul 30 01:23:15.266847 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:23:15.266768    2546 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-serving-cert: secret "maas-api-e2e-mt-671ba06b-serving-cert" not found
Jul 30 01:23:15.266847 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:23:15.266827    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a404f130-bf63-4f08-9b60-8afefbe9a561-maas-api-tls podName:a404f130-bf63-4f08-9b60-8afefbe9a561 nodeName:}" failed. No retries permitted until 2026-07-30 01:23:15.766810905 +0000 UTC m=+2382.989544701 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a404f130-bf63-4f08-9b60-8afefbe9a561-maas-api-tls") pod "maas-api-e2e-mt-671ba06b-585685dcd-5wbnd" (UID: "a404f130-bf63-4f08-9b60-8afefbe9a561") : secret "maas-api-e2e-mt-671ba06b-serving-cert" not found
Jul 30 01:23:15.267321 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.267303    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a404f130-bf63-4f08-9b60-8afefbe9a561-openshift-service-ca\") pod \"maas-api-e2e-mt-671ba06b-585685dcd-5wbnd\" (UID: \"a404f130-bf63-4f08-9b60-8afefbe9a561\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd"
Jul 30 01:23:15.277016 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.276986    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnrnm\" (UniqueName: \"kubernetes.io/projected/a404f130-bf63-4f08-9b60-8afefbe9a561-kube-api-access-cnrnm\") pod \"maas-api-e2e-mt-671ba06b-585685dcd-5wbnd\" (UID: \"a404f130-bf63-4f08-9b60-8afefbe9a561\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd"
Jul 30 01:23:15.771342 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.771303    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a404f130-bf63-4f08-9b60-8afefbe9a561-maas-api-tls\") pod \"maas-api-e2e-mt-671ba06b-585685dcd-5wbnd\" (UID: \"a404f130-bf63-4f08-9b60-8afefbe9a561\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd"
Jul 30 01:23:15.773897 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.773834    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a404f130-bf63-4f08-9b60-8afefbe9a561-maas-api-tls\") pod \"maas-api-e2e-mt-671ba06b-585685dcd-5wbnd\" (UID: \"a404f130-bf63-4f08-9b60-8afefbe9a561\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd"
Jul 30 01:23:15.836260 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.836224    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn"]
Jul 30 01:23:15.839759 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.839742    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn"
Jul 30 01:23:15.842329 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.842309    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-671ba06b-dockercfg-bfd5p\""
Jul 30 01:23:15.842445 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.842312    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-671ba06b\""
Jul 30 01:23:15.853538 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.853040    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn"]
Jul 30 01:23:15.883200 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.883171    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s"]
Jul 30 01:23:15.889783 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.889756    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s"
Jul 30 01:23:15.896291 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.896262    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s"]
Jul 30 01:23:15.957065 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.957040    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd"
Jul 30 01:23:15.973886 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.973764    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7-plugins-config-volume\") pod \"payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn\" (UID: \"7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7\") " pod="openshift-ingress/payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn"
Jul 30 01:23:15.974176 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:15.974154    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4p2t\" (UniqueName: \"kubernetes.io/projected/7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7-kube-api-access-c4p2t\") pod \"payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn\" (UID: \"7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7\") " pod="openshift-ingress/payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn"
Jul 30 01:23:16.075166 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:16.075129    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d9449e2-d1b1-424a-87fc-6f21ccc348bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s\" (UID: \"3d9449e2-d1b1-424a-87fc-6f21ccc348bf\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s"
Jul 30 01:23:16.075626 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:16.075233    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4p2t\" (UniqueName: \"kubernetes.io/projected/7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7-kube-api-access-c4p2t\") pod \"payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn\" (UID: \"7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7\") " pod="openshift-ingress/payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn"
Jul 30 01:23:16.075626 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:16.075314    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tbcm\" (UniqueName: \"kubernetes.io/projected/3d9449e2-d1b1-424a-87fc-6f21ccc348bf-kube-api-access-4tbcm\") pod \"payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s\" (UID: \"3d9449e2-d1b1-424a-87fc-6f21ccc348bf\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s"
Jul 30 01:23:16.075626 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:16.075354    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7-plugins-config-volume\") pod \"payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn\" (UID: \"7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7\") " pod="openshift-ingress/payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn"
Jul 30 01:23:16.076180 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:16.076155    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7-plugins-config-volume\") pod \"payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn\" (UID: \"7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7\") " pod="openshift-ingress/payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn"
Jul 30 01:23:16.083946 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:16.083921    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4p2t\" (UniqueName: \"kubernetes.io/projected/7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7-kube-api-access-c4p2t\") pod \"payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn\" (UID: \"7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7\") " pod="openshift-ingress/payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn"
Jul 30 01:23:16.092951 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:16.092925    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd"]
Jul 30 01:23:16.100051 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:23:16.100020    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda404f130_bf63_4f08_9b60_8afefbe9a561.slice/crio-02222b8a138845fbb85bf72ca29c39d9dd3d6f421d1d5ac5208e89844bef0308 WatchSource:0}: Error finding container 02222b8a138845fbb85bf72ca29c39d9dd3d6f421d1d5ac5208e89844bef0308: Status 404 returned error can't find the container with id 02222b8a138845fbb85bf72ca29c39d9dd3d6f421d1d5ac5208e89844bef0308
Jul 30 01:23:16.101389 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:16.101367    2546 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 01:23:16.159072 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:16.159043    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn"
Jul 30 01:23:16.176480 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:16.176456    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tbcm\" (UniqueName: \"kubernetes.io/projected/3d9449e2-d1b1-424a-87fc-6f21ccc348bf-kube-api-access-4tbcm\") pod \"payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s\" (UID: \"3d9449e2-d1b1-424a-87fc-6f21ccc348bf\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s"
Jul 30 01:23:16.176607 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:16.176573    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d9449e2-d1b1-424a-87fc-6f21ccc348bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s\" (UID: \"3d9449e2-d1b1-424a-87fc-6f21ccc348bf\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s"
Jul 30 01:23:16.177263 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:16.177245    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d9449e2-d1b1-424a-87fc-6f21ccc348bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s\" (UID: \"3d9449e2-d1b1-424a-87fc-6f21ccc348bf\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s"
Jul 30 01:23:16.184344 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:16.184324    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tbcm\" (UniqueName: \"kubernetes.io/projected/3d9449e2-d1b1-424a-87fc-6f21ccc348bf-kube-api-access-4tbcm\") pod \"payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s\" (UID: \"3d9449e2-d1b1-424a-87fc-6f21ccc348bf\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s"
Jul 30 01:23:16.197808 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:16.197777    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd" event={"ID":"a404f130-bf63-4f08-9b60-8afefbe9a561","Type":"ContainerStarted","Data":"02222b8a138845fbb85bf72ca29c39d9dd3d6f421d1d5ac5208e89844bef0308"}
Jul 30 01:23:16.201999 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:16.201974    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s"
Jul 30 01:23:16.313217 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:16.313186    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn"]
Jul 30 01:23:16.317099 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:23:16.316963    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d73fc33_5c5a_40bb_a3e8_876b8f93dbb7.slice/crio-ed0f0fb8cf56ab29d2c2b9a1c2943194cd3d5f564392addeae3e86bbcb1d5f1f WatchSource:0}: Error finding container ed0f0fb8cf56ab29d2c2b9a1c2943194cd3d5f564392addeae3e86bbcb1d5f1f: Status 404 returned error can't find the container with id ed0f0fb8cf56ab29d2c2b9a1c2943194cd3d5f564392addeae3e86bbcb1d5f1f
Jul 30 01:23:16.365267 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:16.365235    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s"]
Jul 30 01:23:16.368174 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:23:16.368146    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d9449e2_d1b1_424a_87fc_6f21ccc348bf.slice/crio-811ac872dc5c876db6168e279514bd0a2474b87e67f3eccb44fe45f0cc433054 WatchSource:0}: Error finding container 811ac872dc5c876db6168e279514bd0a2474b87e67f3eccb44fe45f0cc433054: Status 404 returned error can't find the container with id 811ac872dc5c876db6168e279514bd0a2474b87e67f3eccb44fe45f0cc433054
Jul 30 01:23:17.204742 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:17.204704    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn" event={"ID":"7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7","Type":"ContainerStarted","Data":"dbc9ad75f003a3f5cbb04608b6f088dcbaa493d9bdb393e2ffb6363c7b768c17"}
Jul 30 01:23:17.205287 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:17.204768    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn" event={"ID":"7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7","Type":"ContainerStarted","Data":"ed0f0fb8cf56ab29d2c2b9a1c2943194cd3d5f564392addeae3e86bbcb1d5f1f"}
Jul 30 01:23:17.205287 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:17.204802    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn"
Jul 30 01:23:17.206337 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:17.206312    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd" event={"ID":"a404f130-bf63-4f08-9b60-8afefbe9a561","Type":"ContainerStarted","Data":"254ce462948211f53833ca1759b564a90f0ac9a5e12c9dea6a4d0f112007bd52"}
Jul 30 01:23:17.206427 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:17.206385    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd"
Jul 30 01:23:17.207971 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:17.207936    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s" event={"ID":"3d9449e2-d1b1-424a-87fc-6f21ccc348bf","Type":"ContainerStarted","Data":"6e68a4e79af104c310e8eb4f3634bb5716874228f7e3b156fd371af5de70933a"}
Jul 30 01:23:17.208059 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:17.207982    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s" event={"ID":"3d9449e2-d1b1-424a-87fc-6f21ccc348bf","Type":"ContainerStarted","Data":"811ac872dc5c876db6168e279514bd0a2474b87e67f3eccb44fe45f0cc433054"}
Jul 30 01:23:17.208107 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:17.208076    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s"
Jul 30 01:23:17.221644 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:17.221599    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn" podStartSLOduration=2.221585203 podStartE2EDuration="2.221585203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:23:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:23:17.221067763 +0000 UTC m=+2384.443801574" watchObservedRunningTime="2026-07-30 01:23:17.221585203 +0000 UTC m=+2384.444319021"
Jul 30 01:23:17.239548 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:17.239479    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd" podStartSLOduration=1.7657900340000001 podStartE2EDuration="2.239465666s" totalImagesPullingTime="473.675632ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:23:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:23:17.237667179 +0000 UTC m=+2384.460400999" watchObservedRunningTime="2026-07-30 01:23:17.239465666 +0000 UTC m=+2384.462199483"
Jul 30 01:23:23.219723 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:23.219692    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd"
Jul 30 01:23:23.242225 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:23.242175    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s" podStartSLOduration=8.242160571 podStartE2EDuration="8.242160571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:23:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:23:17.253620992 +0000 UTC m=+2384.476354811" watchObservedRunningTime="2026-07-30 01:23:23.242160571 +0000 UTC m=+2390.464894389"
Jul 30 01:23:28.215732 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:28.215704    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s"
Jul 30 01:23:28.216191 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:28.216010    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn"
Jul 30 01:23:33.380863 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:33.380830    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 01:23:33.383233 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:23:33.383211    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 01:24:23.497497 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:23.497461    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-671ba06b-hhfwd"]
Jul 30 01:24:23.501636 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:23.501615    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-671ba06b-hhfwd"
Jul 30 01:24:23.504483 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:23.504450    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8cqh6\""
Jul 30 01:24:23.506787 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:23.506725    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-671ba06b-hhfwd"]
Jul 30 01:24:23.572513 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:23.572483    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/194a0b1b-1324-4cdb-a4d2-7f7e673cc018-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-671ba06b-hhfwd\" (UID: \"194a0b1b-1324-4cdb-a4d2-7f7e673cc018\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-671ba06b-hhfwd"
Jul 30 01:24:23.673666 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:23.673634    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/194a0b1b-1324-4cdb-a4d2-7f7e673cc018-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-671ba06b-hhfwd\" (UID: \"194a0b1b-1324-4cdb-a4d2-7f7e673cc018\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-671ba06b-hhfwd"
Jul 30 01:24:23.674254 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:23.674235    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/194a0b1b-1324-4cdb-a4d2-7f7e673cc018-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-671ba06b-hhfwd\" (UID: \"194a0b1b-1324-4cdb-a4d2-7f7e673cc018\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-671ba06b-hhfwd"
Jul 30 01:24:23.814584 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:23.814470    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-671ba06b-hhfwd"
Jul 30 01:24:23.957086 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:23.957060    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-671ba06b-hhfwd"]
Jul 30 01:24:23.959107 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:24:23.959079    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod194a0b1b_1324_4cdb_a4d2_7f7e673cc018.slice/crio-48201f3f23c162af9c3da4c4240d1487d09d28241cb3999ae7f72878e509f2b5 WatchSource:0}: Error finding container 48201f3f23c162af9c3da4c4240d1487d09d28241cb3999ae7f72878e509f2b5: Status 404 returned error can't find the container with id 48201f3f23c162af9c3da4c4240d1487d09d28241cb3999ae7f72878e509f2b5
Jul 30 01:24:24.551422 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:24.551386    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-671ba06b-hhfwd" event={"ID":"194a0b1b-1324-4cdb-a4d2-7f7e673cc018","Type":"ContainerStarted","Data":"48201f3f23c162af9c3da4c4240d1487d09d28241cb3999ae7f72878e509f2b5"}
Jul 30 01:24:28.575610 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:28.575568    2546 generic.go:356] "Generic (PLEG): container finished" podID="194a0b1b-1324-4cdb-a4d2-7f7e673cc018" containerID="1539eeb33996ce8cde87fcb7d52557a19c291b2737af9498bc77fce6cb05ed9d" exitCode=0
Jul 30 01:24:28.576001 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:28.575637    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-671ba06b-hhfwd" event={"ID":"194a0b1b-1324-4cdb-a4d2-7f7e673cc018","Type":"ContainerDied","Data":"1539eeb33996ce8cde87fcb7d52557a19c291b2737af9498bc77fce6cb05ed9d"}
Jul 30 01:24:29.719430 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:29.719402    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-671ba06b-hhfwd"
Jul 30 01:24:29.835463 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:29.835388    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/194a0b1b-1324-4cdb-a4d2-7f7e673cc018-maas-api-service-ca\") pod \"194a0b1b-1324-4cdb-a4d2-7f7e673cc018\" (UID: \"194a0b1b-1324-4cdb-a4d2-7f7e673cc018\") "
Jul 30 01:24:29.835755 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:29.835734    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/194a0b1b-1324-4cdb-a4d2-7f7e673cc018-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "194a0b1b-1324-4cdb-a4d2-7f7e673cc018" (UID: "194a0b1b-1324-4cdb-a4d2-7f7e673cc018"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:24:29.936909 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:29.936876    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/194a0b1b-1324-4cdb-a4d2-7f7e673cc018-maas-api-service-ca\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:24:30.588838 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:30.588805    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-671ba06b-hhfwd" event={"ID":"194a0b1b-1324-4cdb-a4d2-7f7e673cc018","Type":"ContainerDied","Data":"48201f3f23c162af9c3da4c4240d1487d09d28241cb3999ae7f72878e509f2b5"}
Jul 30 01:24:30.588838 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:30.588833    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-671ba06b-hhfwd"
Jul 30 01:24:30.589114 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:30.588857    2546 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="48201f3f23c162af9c3da4c4240d1487d09d28241cb3999ae7f72878e509f2b5"
Jul 30 01:24:38.990872 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:38.990835    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd"]
Jul 30 01:24:38.991340 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:38.991291    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd" podUID="a404f130-bf63-4f08-9b60-8afefbe9a561" containerName="maas-api" containerID="cri-o://254ce462948211f53833ca1759b564a90f0ac9a5e12c9dea6a4d0f112007bd52" gracePeriod=30
Jul 30 01:24:39.050684 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.050641    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn"]
Jul 30 01:24:39.050948 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.050918    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn" podUID="7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7" containerName="payload-processing" containerID="cri-o://dbc9ad75f003a3f5cbb04608b6f088dcbaa493d9bdb393e2ffb6363c7b768c17" gracePeriod=30
Jul 30 01:24:39.058373 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.058344    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s"]
Jul 30 01:24:39.058596 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.058571    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s" podUID="3d9449e2-d1b1-424a-87fc-6f21ccc348bf" containerName="payload-pre-processing" containerID="cri-o://6e68a4e79af104c310e8eb4f3634bb5716874228f7e3b156fd371af5de70933a" gracePeriod=30
Jul 30 01:24:39.191320 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.191285    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-671ba06b-hhfwd"]
Jul 30 01:24:39.196304 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.196270    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-671ba06b-hhfwd"]
Jul 30 01:24:39.331775 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.331746    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn"
Jul 30 01:24:39.360059 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.360035    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s"
Jul 30 01:24:39.360204 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.360035    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="194a0b1b-1324-4cdb-a4d2-7f7e673cc018" path="/var/lib/kubelet/pods/194a0b1b-1324-4cdb-a4d2-7f7e673cc018/volumes"
Jul 30 01:24:39.369990 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.369967    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd"
Jul 30 01:24:39.419398 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.419366    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4p2t\" (UniqueName: \"kubernetes.io/projected/7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7-kube-api-access-c4p2t\") pod \"7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7\" (UID: \"7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7\") "
Jul 30 01:24:39.419610 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.419447    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7-plugins-config-volume\") pod \"7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7\" (UID: \"7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7\") "
Jul 30 01:24:39.419821 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.419799    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7" (UID: "7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:24:39.421454 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.421423    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7-kube-api-access-c4p2t" (OuterVolumeSpecName: "kube-api-access-c4p2t") pod "7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7" (UID: "7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7"). InnerVolumeSpecName "kube-api-access-c4p2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:24:39.520712 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.520632    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d9449e2-d1b1-424a-87fc-6f21ccc348bf-plugins-config-volume\") pod \"3d9449e2-d1b1-424a-87fc-6f21ccc348bf\" (UID: \"3d9449e2-d1b1-424a-87fc-6f21ccc348bf\") "
Jul 30 01:24:39.520712 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.520673    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tbcm\" (UniqueName: \"kubernetes.io/projected/3d9449e2-d1b1-424a-87fc-6f21ccc348bf-kube-api-access-4tbcm\") pod \"3d9449e2-d1b1-424a-87fc-6f21ccc348bf\" (UID: \"3d9449e2-d1b1-424a-87fc-6f21ccc348bf\") "
Jul 30 01:24:39.520712 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.520699    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnrnm\" (UniqueName: \"kubernetes.io/projected/a404f130-bf63-4f08-9b60-8afefbe9a561-kube-api-access-cnrnm\") pod \"a404f130-bf63-4f08-9b60-8afefbe9a561\" (UID: \"a404f130-bf63-4f08-9b60-8afefbe9a561\") "
Jul 30 01:24:39.521011 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.520720    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a404f130-bf63-4f08-9b60-8afefbe9a561-maas-api-tls\") pod \"a404f130-bf63-4f08-9b60-8afefbe9a561\" (UID: \"a404f130-bf63-4f08-9b60-8afefbe9a561\") "
Jul 30 01:24:39.521011 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.520810    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a404f130-bf63-4f08-9b60-8afefbe9a561-openshift-service-ca\") pod \"a404f130-bf63-4f08-9b60-8afefbe9a561\" (UID: \"a404f130-bf63-4f08-9b60-8afefbe9a561\") "
Jul 30 01:24:39.521133 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.521081    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4p2t\" (UniqueName: \"kubernetes.io/projected/7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7-kube-api-access-c4p2t\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:24:39.521133 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.521103    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:24:39.521248 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.521173    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d9449e2-d1b1-424a-87fc-6f21ccc348bf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3d9449e2-d1b1-424a-87fc-6f21ccc348bf" (UID: "3d9449e2-d1b1-424a-87fc-6f21ccc348bf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:24:39.521248 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.521208    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a404f130-bf63-4f08-9b60-8afefbe9a561-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a404f130-bf63-4f08-9b60-8afefbe9a561" (UID: "a404f130-bf63-4f08-9b60-8afefbe9a561"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:24:39.522770 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.522746    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a404f130-bf63-4f08-9b60-8afefbe9a561-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a404f130-bf63-4f08-9b60-8afefbe9a561" (UID: "a404f130-bf63-4f08-9b60-8afefbe9a561"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:24:39.522929 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.522847    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a404f130-bf63-4f08-9b60-8afefbe9a561-kube-api-access-cnrnm" (OuterVolumeSpecName: "kube-api-access-cnrnm") pod "a404f130-bf63-4f08-9b60-8afefbe9a561" (UID: "a404f130-bf63-4f08-9b60-8afefbe9a561"). InnerVolumeSpecName "kube-api-access-cnrnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:24:39.522929 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.522868    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d9449e2-d1b1-424a-87fc-6f21ccc348bf-kube-api-access-4tbcm" (OuterVolumeSpecName: "kube-api-access-4tbcm") pod "3d9449e2-d1b1-424a-87fc-6f21ccc348bf" (UID: "3d9449e2-d1b1-424a-87fc-6f21ccc348bf"). InnerVolumeSpecName "kube-api-access-4tbcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:24:39.622305 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.622269    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnrnm\" (UniqueName: \"kubernetes.io/projected/a404f130-bf63-4f08-9b60-8afefbe9a561-kube-api-access-cnrnm\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:24:39.622305 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.622303    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a404f130-bf63-4f08-9b60-8afefbe9a561-maas-api-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:24:39.622495 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.622318    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a404f130-bf63-4f08-9b60-8afefbe9a561-openshift-service-ca\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:24:39.622495 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.622330    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d9449e2-d1b1-424a-87fc-6f21ccc348bf-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:24:39.622495 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.622343    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tbcm\" (UniqueName: \"kubernetes.io/projected/3d9449e2-d1b1-424a-87fc-6f21ccc348bf-kube-api-access-4tbcm\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:24:39.639939 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.639908    2546 generic.go:356] "Generic (PLEG): container finished" podID="7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7" containerID="dbc9ad75f003a3f5cbb04608b6f088dcbaa493d9bdb393e2ffb6363c7b768c17" exitCode=0
Jul 30 01:24:39.640079 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.639951    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn"
Jul 30 01:24:39.640079 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.639995    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn" event={"ID":"7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7","Type":"ContainerDied","Data":"dbc9ad75f003a3f5cbb04608b6f088dcbaa493d9bdb393e2ffb6363c7b768c17"}
Jul 30 01:24:39.640079 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.640053    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn" event={"ID":"7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7","Type":"ContainerDied","Data":"ed0f0fb8cf56ab29d2c2b9a1c2943194cd3d5f564392addeae3e86bbcb1d5f1f"}
Jul 30 01:24:39.640257 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.640084    2546 scope.go:117] "RemoveContainer" containerID="dbc9ad75f003a3f5cbb04608b6f088dcbaa493d9bdb393e2ffb6363c7b768c17"
Jul 30 01:24:39.641280 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.641260    2546 generic.go:356] "Generic (PLEG): container finished" podID="a404f130-bf63-4f08-9b60-8afefbe9a561" containerID="254ce462948211f53833ca1759b564a90f0ac9a5e12c9dea6a4d0f112007bd52" exitCode=0
Jul 30 01:24:39.641360 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.641297    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd"
Jul 30 01:24:39.641360 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.641339    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd" event={"ID":"a404f130-bf63-4f08-9b60-8afefbe9a561","Type":"ContainerDied","Data":"254ce462948211f53833ca1759b564a90f0ac9a5e12c9dea6a4d0f112007bd52"}
Jul 30 01:24:39.641455 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.641397    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd" event={"ID":"a404f130-bf63-4f08-9b60-8afefbe9a561","Type":"ContainerDied","Data":"02222b8a138845fbb85bf72ca29c39d9dd3d6f421d1d5ac5208e89844bef0308"}
Jul 30 01:24:39.642695 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.642671    2546 generic.go:356] "Generic (PLEG): container finished" podID="3d9449e2-d1b1-424a-87fc-6f21ccc348bf" containerID="6e68a4e79af104c310e8eb4f3634bb5716874228f7e3b156fd371af5de70933a" exitCode=0
Jul 30 01:24:39.642816 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.642703    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s" event={"ID":"3d9449e2-d1b1-424a-87fc-6f21ccc348bf","Type":"ContainerDied","Data":"6e68a4e79af104c310e8eb4f3634bb5716874228f7e3b156fd371af5de70933a"}
Jul 30 01:24:39.642816 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.642755    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s" event={"ID":"3d9449e2-d1b1-424a-87fc-6f21ccc348bf","Type":"ContainerDied","Data":"811ac872dc5c876db6168e279514bd0a2474b87e67f3eccb44fe45f0cc433054"}
Jul 30 01:24:39.642816 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.642715    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s"
Jul 30 01:24:39.651167 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.651091    2546 scope.go:117] "RemoveContainer" containerID="dbc9ad75f003a3f5cbb04608b6f088dcbaa493d9bdb393e2ffb6363c7b768c17"
Jul 30 01:24:39.651462 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:24:39.651441    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbc9ad75f003a3f5cbb04608b6f088dcbaa493d9bdb393e2ffb6363c7b768c17\": container with ID starting with dbc9ad75f003a3f5cbb04608b6f088dcbaa493d9bdb393e2ffb6363c7b768c17 not found: ID does not exist" containerID="dbc9ad75f003a3f5cbb04608b6f088dcbaa493d9bdb393e2ffb6363c7b768c17"
Jul 30 01:24:39.651559 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.651468    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbc9ad75f003a3f5cbb04608b6f088dcbaa493d9bdb393e2ffb6363c7b768c17"} err="failed to get container status \"dbc9ad75f003a3f5cbb04608b6f088dcbaa493d9bdb393e2ffb6363c7b768c17\": rpc error: code = NotFound desc = could not find container \"dbc9ad75f003a3f5cbb04608b6f088dcbaa493d9bdb393e2ffb6363c7b768c17\": container with ID starting with dbc9ad75f003a3f5cbb04608b6f088dcbaa493d9bdb393e2ffb6363c7b768c17 not found: ID does not exist"
Jul 30 01:24:39.651559 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.651483    2546 scope.go:117] "RemoveContainer" containerID="254ce462948211f53833ca1759b564a90f0ac9a5e12c9dea6a4d0f112007bd52"
Jul 30 01:24:39.659371 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.659354    2546 scope.go:117] "RemoveContainer" containerID="254ce462948211f53833ca1759b564a90f0ac9a5e12c9dea6a4d0f112007bd52"
Jul 30 01:24:39.659586 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:24:39.659569    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"254ce462948211f53833ca1759b564a90f0ac9a5e12c9dea6a4d0f112007bd52\": container with ID starting with 254ce462948211f53833ca1759b564a90f0ac9a5e12c9dea6a4d0f112007bd52 not found: ID does not exist" containerID="254ce462948211f53833ca1759b564a90f0ac9a5e12c9dea6a4d0f112007bd52"
Jul 30 01:24:39.659648 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.659591    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"254ce462948211f53833ca1759b564a90f0ac9a5e12c9dea6a4d0f112007bd52"} err="failed to get container status \"254ce462948211f53833ca1759b564a90f0ac9a5e12c9dea6a4d0f112007bd52\": rpc error: code = NotFound desc = could not find container \"254ce462948211f53833ca1759b564a90f0ac9a5e12c9dea6a4d0f112007bd52\": container with ID starting with 254ce462948211f53833ca1759b564a90f0ac9a5e12c9dea6a4d0f112007bd52 not found: ID does not exist"
Jul 30 01:24:39.659648 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.659607    2546 scope.go:117] "RemoveContainer" containerID="6e68a4e79af104c310e8eb4f3634bb5716874228f7e3b156fd371af5de70933a"
Jul 30 01:24:39.667164 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.667139    2546 scope.go:117] "RemoveContainer" containerID="6e68a4e79af104c310e8eb4f3634bb5716874228f7e3b156fd371af5de70933a"
Jul 30 01:24:39.667386 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:24:39.667369    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e68a4e79af104c310e8eb4f3634bb5716874228f7e3b156fd371af5de70933a\": container with ID starting with 6e68a4e79af104c310e8eb4f3634bb5716874228f7e3b156fd371af5de70933a not found: ID does not exist" containerID="6e68a4e79af104c310e8eb4f3634bb5716874228f7e3b156fd371af5de70933a"
Jul 30 01:24:39.667441 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.667390    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e68a4e79af104c310e8eb4f3634bb5716874228f7e3b156fd371af5de70933a"} err="failed to get container status \"6e68a4e79af104c310e8eb4f3634bb5716874228f7e3b156fd371af5de70933a\": rpc error: code = NotFound desc = could not find container \"6e68a4e79af104c310e8eb4f3634bb5716874228f7e3b156fd371af5de70933a\": container with ID starting with 6e68a4e79af104c310e8eb4f3634bb5716874228f7e3b156fd371af5de70933a not found: ID does not exist"
Jul 30 01:24:39.668134 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.668115    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd"]
Jul 30 01:24:39.671911 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.671890    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-671ba06b-585685dcd-5wbnd"]
Jul 30 01:24:39.681971 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.681952    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn"]
Jul 30 01:24:39.687891 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.687871    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-671ba06b-6ff9c5f5c8-6fbcn"]
Jul 30 01:24:39.698555 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.698504    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s"]
Jul 30 01:24:39.704739 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:39.704716    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-671ba06b-74776589d8-zgn7s"]
Jul 30 01:24:41.361497 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:41.361450    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d9449e2-d1b1-424a-87fc-6f21ccc348bf" path="/var/lib/kubelet/pods/3d9449e2-d1b1-424a-87fc-6f21ccc348bf/volumes"
Jul 30 01:24:41.362067 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:41.362042    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7" path="/var/lib/kubelet/pods/7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7/volumes"
Jul 30 01:24:41.362635 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:24:41.362610    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a404f130-bf63-4f08-9b60-8afefbe9a561" path="/var/lib/kubelet/pods/a404f130-bf63-4f08-9b60-8afefbe9a561/volumes"
Jul 30 01:26:53.242655 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.242624    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv"]
Jul 30 01:26:53.245085 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.243001    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="194a0b1b-1324-4cdb-a4d2-7f7e673cc018" containerName="revoke-keys"
Jul 30 01:26:53.245085 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.243012    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="194a0b1b-1324-4cdb-a4d2-7f7e673cc018" containerName="revoke-keys"
Jul 30 01:26:53.245085 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.243027    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7" containerName="payload-processing"
Jul 30 01:26:53.245085 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.243033    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7" containerName="payload-processing"
Jul 30 01:26:53.245085 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.243043    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a404f130-bf63-4f08-9b60-8afefbe9a561" containerName="maas-api"
Jul 30 01:26:53.245085 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.243048    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="a404f130-bf63-4f08-9b60-8afefbe9a561" containerName="maas-api"
Jul 30 01:26:53.245085 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.243060    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d9449e2-d1b1-424a-87fc-6f21ccc348bf" containerName="payload-pre-processing"
Jul 30 01:26:53.245085 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.243066    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d9449e2-d1b1-424a-87fc-6f21ccc348bf" containerName="payload-pre-processing"
Jul 30 01:26:53.245085 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.243125    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d9449e2-d1b1-424a-87fc-6f21ccc348bf" containerName="payload-pre-processing"
Jul 30 01:26:53.245085 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.243131    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="194a0b1b-1324-4cdb-a4d2-7f7e673cc018" containerName="revoke-keys"
Jul 30 01:26:53.245085 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.243140    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d73fc33-5c5a-40bb-a3e8-876b8f93dbb7" containerName="payload-processing"
Jul 30 01:26:53.245085 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.243146    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="a404f130-bf63-4f08-9b60-8afefbe9a561" containerName="maas-api"
Jul 30 01:26:53.246701 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.246681    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv"
Jul 30 01:26:53.249197 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.249179    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-6ce027-serving-cert\""
Jul 30 01:26:53.258080 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.258054    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv"]
Jul 30 01:26:53.346065 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.346035    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-982gp\" (UniqueName: \"kubernetes.io/projected/0d9148ec-84c7-48af-918d-8abd007f9a8c-kube-api-access-982gp\") pod \"maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv\" (UID: \"0d9148ec-84c7-48af-918d-8abd007f9a8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv"
Jul 30 01:26:53.346254 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.346132    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d9148ec-84c7-48af-918d-8abd007f9a8c-maas-api-tls\") pod \"maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv\" (UID: \"0d9148ec-84c7-48af-918d-8abd007f9a8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv"
Jul 30 01:26:53.346254 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.346160    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d9148ec-84c7-48af-918d-8abd007f9a8c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv\" (UID: \"0d9148ec-84c7-48af-918d-8abd007f9a8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv"
Jul 30 01:26:53.447663 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.447627    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d9148ec-84c7-48af-918d-8abd007f9a8c-maas-api-tls\") pod \"maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv\" (UID: \"0d9148ec-84c7-48af-918d-8abd007f9a8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv"
Jul 30 01:26:53.447841 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.447668    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d9148ec-84c7-48af-918d-8abd007f9a8c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv\" (UID: \"0d9148ec-84c7-48af-918d-8abd007f9a8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv"
Jul 30 01:26:53.447841 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.447747    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-982gp\" (UniqueName: \"kubernetes.io/projected/0d9148ec-84c7-48af-918d-8abd007f9a8c-kube-api-access-982gp\") pod \"maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv\" (UID: \"0d9148ec-84c7-48af-918d-8abd007f9a8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv"
Jul 30 01:26:53.447841 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:26:53.447762    2546 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-serving-cert: secret "maas-api-e2e-inf-a-6ce027-serving-cert" not found
Jul 30 01:26:53.447841 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:26:53.447833    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d9148ec-84c7-48af-918d-8abd007f9a8c-maas-api-tls podName:0d9148ec-84c7-48af-918d-8abd007f9a8c nodeName:}" failed. No retries permitted until 2026-07-30 01:26:53.947811443 +0000 UTC m=+2601.170545259 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0d9148ec-84c7-48af-918d-8abd007f9a8c-maas-api-tls") pod "maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv" (UID: "0d9148ec-84c7-48af-918d-8abd007f9a8c") : secret "maas-api-e2e-inf-a-6ce027-serving-cert" not found
Jul 30 01:26:53.448307 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.448288    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d9148ec-84c7-48af-918d-8abd007f9a8c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv\" (UID: \"0d9148ec-84c7-48af-918d-8abd007f9a8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv"
Jul 30 01:26:53.456983 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.456959    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-982gp\" (UniqueName: \"kubernetes.io/projected/0d9148ec-84c7-48af-918d-8abd007f9a8c-kube-api-access-982gp\") pod \"maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv\" (UID: \"0d9148ec-84c7-48af-918d-8abd007f9a8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv"
Jul 30 01:26:53.826493 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.826455    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc"]
Jul 30 01:26:53.830101 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.830084    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc"
Jul 30 01:26:53.832985 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.832934    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-6ce027-dockercfg-ngbq8\""
Jul 30 01:26:53.832985 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.832969    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-6ce027\""
Jul 30 01:26:53.839628 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.839606    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc"]
Jul 30 01:26:53.851602 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.851580    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc\" (UID: \"e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc"
Jul 30 01:26:53.851723 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.851611    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2dbz\" (UniqueName: \"kubernetes.io/projected/e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9-kube-api-access-k2dbz\") pod \"payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc\" (UID: \"e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc"
Jul 30 01:26:53.858918 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.858890    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h"]
Jul 30 01:26:53.862366 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.862349    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h"
Jul 30 01:26:53.874074 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.874053    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h"]
Jul 30 01:26:53.952593 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.952560    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc\" (UID: \"e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc"
Jul 30 01:26:53.952593 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.952603    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97gd5\" (UniqueName: \"kubernetes.io/projected/f66b1606-c471-4488-849d-996048b6c1d9-kube-api-access-97gd5\") pod \"payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h\" (UID: \"f66b1606-c471-4488-849d-996048b6c1d9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h"
Jul 30 01:26:53.952905 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.952627    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2dbz\" (UniqueName: \"kubernetes.io/projected/e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9-kube-api-access-k2dbz\") pod \"payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc\" (UID: \"e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc"
Jul 30 01:26:53.952905 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.952706    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d9148ec-84c7-48af-918d-8abd007f9a8c-maas-api-tls\") pod \"maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv\" (UID: \"0d9148ec-84c7-48af-918d-8abd007f9a8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv"
Jul 30 01:26:53.952905 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.952771    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f66b1606-c471-4488-849d-996048b6c1d9-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h\" (UID: \"f66b1606-c471-4488-849d-996048b6c1d9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h"
Jul 30 01:26:53.953439 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.953420    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc\" (UID: \"e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc"
Jul 30 01:26:53.955279 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.955253    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d9148ec-84c7-48af-918d-8abd007f9a8c-maas-api-tls\") pod \"maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv\" (UID: \"0d9148ec-84c7-48af-918d-8abd007f9a8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv"
Jul 30 01:26:53.960881 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:53.960859    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2dbz\" (UniqueName: \"kubernetes.io/projected/e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9-kube-api-access-k2dbz\") pod \"payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc\" (UID: \"e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc"
Jul 30 01:26:54.053844 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:54.053810    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97gd5\" (UniqueName: \"kubernetes.io/projected/f66b1606-c471-4488-849d-996048b6c1d9-kube-api-access-97gd5\") pod \"payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h\" (UID: \"f66b1606-c471-4488-849d-996048b6c1d9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h"
Jul 30 01:26:54.054027 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:54.053872    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f66b1606-c471-4488-849d-996048b6c1d9-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h\" (UID: \"f66b1606-c471-4488-849d-996048b6c1d9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h"
Jul 30 01:26:54.054639 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:54.054614    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f66b1606-c471-4488-849d-996048b6c1d9-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h\" (UID: \"f66b1606-c471-4488-849d-996048b6c1d9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h"
Jul 30 01:26:54.062576 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:54.062555    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97gd5\" (UniqueName: \"kubernetes.io/projected/f66b1606-c471-4488-849d-996048b6c1d9-kube-api-access-97gd5\") pod \"payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h\" (UID: \"f66b1606-c471-4488-849d-996048b6c1d9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h"
Jul 30 01:26:54.142915 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:54.142827    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc"
Jul 30 01:26:54.159046 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:54.159019    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv"
Jul 30 01:26:54.173377 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:54.173331    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h"
Jul 30 01:26:54.519621 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:54.519444    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc"]
Jul 30 01:26:54.522028 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:26:54.521986    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1dc6e10_b79e_47ff_a07d_b7cc4219c1b9.slice/crio-d01192fc250303fc6ea4248e5347121e76379933d500e74aef5d681331b4a26c WatchSource:0}: Error finding container d01192fc250303fc6ea4248e5347121e76379933d500e74aef5d681331b4a26c: Status 404 returned error can't find the container with id d01192fc250303fc6ea4248e5347121e76379933d500e74aef5d681331b4a26c
Jul 30 01:26:54.561018 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:54.560083    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h"]
Jul 30 01:26:54.561018 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:54.560219    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv"]
Jul 30 01:26:54.567890 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:26:54.567864    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d9148ec_84c7_48af_918d_8abd007f9a8c.slice/crio-869ec035577be3e8243300dd593301f05fab856fca4b3336197aaf5505dea1a4 WatchSource:0}: Error finding container 869ec035577be3e8243300dd593301f05fab856fca4b3336197aaf5505dea1a4: Status 404 returned error can't find the container with id 869ec035577be3e8243300dd593301f05fab856fca4b3336197aaf5505dea1a4
Jul 30 01:26:55.331225 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:55.331186    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv" event={"ID":"0d9148ec-84c7-48af-918d-8abd007f9a8c","Type":"ContainerStarted","Data":"e28d1f4b5d2e4369d906325fa8e84ebaac39ed5e8b142d8500ebe6401e8c4e03"}
Jul 30 01:26:55.331434 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:55.331256    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv" event={"ID":"0d9148ec-84c7-48af-918d-8abd007f9a8c","Type":"ContainerStarted","Data":"869ec035577be3e8243300dd593301f05fab856fca4b3336197aaf5505dea1a4"}
Jul 30 01:26:55.331434 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:55.331403    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv"
Jul 30 01:26:55.332986 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:55.332957    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc" event={"ID":"e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9","Type":"ContainerStarted","Data":"f16067b5e40abf9b635a5db533bd8c3fa18ddacd9314e6637add475e6d727f9a"}
Jul 30 01:26:55.333130 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:55.333018    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc" event={"ID":"e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9","Type":"ContainerStarted","Data":"d01192fc250303fc6ea4248e5347121e76379933d500e74aef5d681331b4a26c"}
Jul 30 01:26:55.333130 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:55.333049    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc"
Jul 30 01:26:55.334489 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:55.334463    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h" event={"ID":"f66b1606-c471-4488-849d-996048b6c1d9","Type":"ContainerStarted","Data":"c775289c227059c9cc8fd6f456d6a2253020aebf9a2261c4fcfa95afb8c7095b"}
Jul 30 01:26:55.334609 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:55.334510    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h" event={"ID":"f66b1606-c471-4488-849d-996048b6c1d9","Type":"ContainerStarted","Data":"85a3cc9d96df5c3faee7d450ce1aae9c632ee3061c240713a54187b7889cfbf0"}
Jul 30 01:26:55.334658 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:55.334639    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h"
Jul 30 01:26:55.348213 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:55.348171    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv" podStartSLOduration=1.9868514099999999 podStartE2EDuration="2.348158927s" totalImagesPullingTime="361.307517ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:26:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:26:55.347555477 +0000 UTC m=+2602.570289296" watchObservedRunningTime="2026-07-30 01:26:55.348158927 +0000 UTC m=+2602.570892744"
Jul 30 01:26:55.365457 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:55.365412    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h" podStartSLOduration=2.365398245 podStartE2EDuration="2.365398245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:26:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:26:55.363965821 +0000 UTC m=+2602.586699638" watchObservedRunningTime="2026-07-30 01:26:55.365398245 +0000 UTC m=+2602.588132063"
Jul 30 01:26:55.382229 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:26:55.382185    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc" podStartSLOduration=2.382171827 podStartE2EDuration="2.382171827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:26:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:26:55.379657493 +0000 UTC m=+2602.602391332" watchObservedRunningTime="2026-07-30 01:26:55.382171827 +0000 UTC m=+2602.604905644"
Jul 30 01:27:01.344670 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:01.344644    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv"
Jul 30 01:27:06.341258 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:06.341208    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc"
Jul 30 01:27:06.341783 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:06.341473    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h"
Jul 30 01:27:21.470885 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:21.470850    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w"]
Jul 30 01:27:21.474491 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:21.474467    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w"
Jul 30 01:27:21.477126 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:21.477102    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-b2ddd3-serving-cert\""
Jul 30 01:27:21.486085 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:21.486046    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w"]
Jul 30 01:27:21.602433 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:21.602393    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b9d4e64-064f-4815-8315-c20c3de7a494-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w\" (UID: \"7b9d4e64-064f-4815-8315-c20c3de7a494\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w"
Jul 30 01:27:21.602433 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:21.602434    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b9d4e64-064f-4815-8315-c20c3de7a494-maas-api-tls\") pod \"maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w\" (UID: \"7b9d4e64-064f-4815-8315-c20c3de7a494\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w"
Jul 30 01:27:21.602706 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:21.602582    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj4gj\" (UniqueName: \"kubernetes.io/projected/7b9d4e64-064f-4815-8315-c20c3de7a494-kube-api-access-xj4gj\") pod \"maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w\" (UID: \"7b9d4e64-064f-4815-8315-c20c3de7a494\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w"
Jul 30 01:27:21.703724 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:21.703679    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj4gj\" (UniqueName: \"kubernetes.io/projected/7b9d4e64-064f-4815-8315-c20c3de7a494-kube-api-access-xj4gj\") pod \"maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w\" (UID: \"7b9d4e64-064f-4815-8315-c20c3de7a494\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w"
Jul 30 01:27:21.703952 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:21.703751    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b9d4e64-064f-4815-8315-c20c3de7a494-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w\" (UID: \"7b9d4e64-064f-4815-8315-c20c3de7a494\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w"
Jul 30 01:27:21.703952 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:21.703900    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b9d4e64-064f-4815-8315-c20c3de7a494-maas-api-tls\") pod \"maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w\" (UID: \"7b9d4e64-064f-4815-8315-c20c3de7a494\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w"
Jul 30 01:27:21.704453 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:21.704425    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b9d4e64-064f-4815-8315-c20c3de7a494-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w\" (UID: \"7b9d4e64-064f-4815-8315-c20c3de7a494\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w"
Jul 30 01:27:21.706396 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:21.706369    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b9d4e64-064f-4815-8315-c20c3de7a494-maas-api-tls\") pod \"maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w\" (UID: \"7b9d4e64-064f-4815-8315-c20c3de7a494\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w"
Jul 30 01:27:21.717955 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:21.717933    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj4gj\" (UniqueName: \"kubernetes.io/projected/7b9d4e64-064f-4815-8315-c20c3de7a494-kube-api-access-xj4gj\") pod \"maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w\" (UID: \"7b9d4e64-064f-4815-8315-c20c3de7a494\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w"
Jul 30 01:27:21.787073 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:21.786974    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w"
Jul 30 01:27:21.929218 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:21.929186    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w"]
Jul 30 01:27:21.931800 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:27:21.931771    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b9d4e64_064f_4815_8315_c20c3de7a494.slice/crio-ee164f89ba19dc54d4c91e7341f9d1676640bb8b350a495d1320568c78b8487c WatchSource:0}: Error finding container ee164f89ba19dc54d4c91e7341f9d1676640bb8b350a495d1320568c78b8487c: Status 404 returned error can't find the container with id ee164f89ba19dc54d4c91e7341f9d1676640bb8b350a495d1320568c78b8487c
Jul 30 01:27:22.160297 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.160212    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm"]
Jul 30 01:27:22.165018 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.164990    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm"
Jul 30 01:27:22.167682 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.167660    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-b2ddd3-dockercfg-mtmk5\""
Jul 30 01:27:22.167818 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.167797    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-b2ddd3\""
Jul 30 01:27:22.176165 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.176141    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm"]
Jul 30 01:27:22.179035 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.179007    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829"]
Jul 30 01:27:22.182783 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.182766    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829"
Jul 30 01:27:22.199205 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.199172    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829"]
Jul 30 01:27:22.310919 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.310875    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebdac8f0-a89f-4814-b773-9eff0b728e3a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm\" (UID: \"ebdac8f0-a89f-4814-b773-9eff0b728e3a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm"
Jul 30 01:27:22.311112 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.311014    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztnr4\" (UniqueName: \"kubernetes.io/projected/40a484b8-6192-444f-8703-97ad2b28938f-kube-api-access-ztnr4\") pod \"payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829\" (UID: \"40a484b8-6192-444f-8703-97ad2b28938f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829"
Jul 30 01:27:22.311112 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.311101    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40a484b8-6192-444f-8703-97ad2b28938f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829\" (UID: \"40a484b8-6192-444f-8703-97ad2b28938f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829"
Jul 30 01:27:22.311211 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.311165    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbw82\" (UniqueName: \"kubernetes.io/projected/ebdac8f0-a89f-4814-b773-9eff0b728e3a-kube-api-access-fbw82\") pod \"payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm\" (UID: \"ebdac8f0-a89f-4814-b773-9eff0b728e3a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm"
Jul 30 01:27:22.412638 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.412531    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40a484b8-6192-444f-8703-97ad2b28938f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829\" (UID: \"40a484b8-6192-444f-8703-97ad2b28938f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829"
Jul 30 01:27:22.412638 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.412581    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbw82\" (UniqueName: \"kubernetes.io/projected/ebdac8f0-a89f-4814-b773-9eff0b728e3a-kube-api-access-fbw82\") pod \"payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm\" (UID: \"ebdac8f0-a89f-4814-b773-9eff0b728e3a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm"
Jul 30 01:27:22.412638 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.412623    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebdac8f0-a89f-4814-b773-9eff0b728e3a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm\" (UID: \"ebdac8f0-a89f-4814-b773-9eff0b728e3a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm"
Jul 30 01:27:22.412935 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.412683    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztnr4\" (UniqueName: \"kubernetes.io/projected/40a484b8-6192-444f-8703-97ad2b28938f-kube-api-access-ztnr4\") pod \"payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829\" (UID: \"40a484b8-6192-444f-8703-97ad2b28938f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829"
Jul 30 01:27:22.413506 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.413480    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebdac8f0-a89f-4814-b773-9eff0b728e3a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm\" (UID: \"ebdac8f0-a89f-4814-b773-9eff0b728e3a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm"
Jul 30 01:27:22.413595 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.413517    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40a484b8-6192-444f-8703-97ad2b28938f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829\" (UID: \"40a484b8-6192-444f-8703-97ad2b28938f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829"
Jul 30 01:27:22.420696 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.420671    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztnr4\" (UniqueName: \"kubernetes.io/projected/40a484b8-6192-444f-8703-97ad2b28938f-kube-api-access-ztnr4\") pod \"payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829\" (UID: \"40a484b8-6192-444f-8703-97ad2b28938f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829"
Jul 30 01:27:22.420814 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.420721    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbw82\" (UniqueName: \"kubernetes.io/projected/ebdac8f0-a89f-4814-b773-9eff0b728e3a-kube-api-access-fbw82\") pod \"payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm\" (UID: \"ebdac8f0-a89f-4814-b773-9eff0b728e3a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm"
Jul 30 01:27:22.476205 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.476167    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w" event={"ID":"7b9d4e64-064f-4815-8315-c20c3de7a494","Type":"ContainerStarted","Data":"8f6794a70a87e15c1c5637c235484be691c25af1ff14380916f30e097a06f409"}
Jul 30 01:27:22.476675 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.476221    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w" event={"ID":"7b9d4e64-064f-4815-8315-c20c3de7a494","Type":"ContainerStarted","Data":"ee164f89ba19dc54d4c91e7341f9d1676640bb8b350a495d1320568c78b8487c"}
Jul 30 01:27:22.476675 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.476244    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w"
Jul 30 01:27:22.476675 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.476455    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm"
Jul 30 01:27:22.495424 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.495365    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w" podStartSLOduration=1.115323455 podStartE2EDuration="1.495343672s" totalImagesPullingTime="380.020217ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:27:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:27:22.493573098 +0000 UTC m=+2629.716306922" watchObservedRunningTime="2026-07-30 01:27:22.495343672 +0000 UTC m=+2629.718077490"
Jul 30 01:27:22.497472 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.497447    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829"
Jul 30 01:27:22.635385 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.635280    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm"]
Jul 30 01:27:22.638650 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:27:22.638614    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebdac8f0_a89f_4814_b773_9eff0b728e3a.slice/crio-5913418bdbb04a25c94fa320f9526a0d48015b96ca4639f9a0fa6502ddaec257 WatchSource:0}: Error finding container 5913418bdbb04a25c94fa320f9526a0d48015b96ca4639f9a0fa6502ddaec257: Status 404 returned error can't find the container with id 5913418bdbb04a25c94fa320f9526a0d48015b96ca4639f9a0fa6502ddaec257
Jul 30 01:27:22.669847 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:22.669822    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829"]
Jul 30 01:27:22.680717 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:27:22.680687    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40a484b8_6192_444f_8703_97ad2b28938f.slice/crio-860b8af55915c8e744ca4b0bd9676bf6e274195c35786372caa8e339f33f12db WatchSource:0}: Error finding container 860b8af55915c8e744ca4b0bd9676bf6e274195c35786372caa8e339f33f12db: Status 404 returned error can't find the container with id 860b8af55915c8e744ca4b0bd9676bf6e274195c35786372caa8e339f33f12db
Jul 30 01:27:23.482740 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:23.482701    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm" event={"ID":"ebdac8f0-a89f-4814-b773-9eff0b728e3a","Type":"ContainerStarted","Data":"c71dde941788470b2e75524c2ce99399aa6b3d46b88e6bf1fa7b78131c59dd01"}
Jul 30 01:27:23.483188 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:23.482765    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm" event={"ID":"ebdac8f0-a89f-4814-b773-9eff0b728e3a","Type":"ContainerStarted","Data":"5913418bdbb04a25c94fa320f9526a0d48015b96ca4639f9a0fa6502ddaec257"}
Jul 30 01:27:23.483188 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:23.482823    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm"
Jul 30 01:27:23.484438 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:23.484411    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829" event={"ID":"40a484b8-6192-444f-8703-97ad2b28938f","Type":"ContainerStarted","Data":"cc6ad6cf1ae078167165d5806cbb015de9e09453cf25e73a8ec6b7a41fe1fc0b"}
Jul 30 01:27:23.484622 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:23.484458    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829" event={"ID":"40a484b8-6192-444f-8703-97ad2b28938f","Type":"ContainerStarted","Data":"860b8af55915c8e744ca4b0bd9676bf6e274195c35786372caa8e339f33f12db"}
Jul 30 01:27:23.502327 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:23.502275    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm" podStartSLOduration=1.502259818 podStartE2EDuration="1.502259818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:27:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:27:23.500584193 +0000 UTC m=+2630.723318010" watchObservedRunningTime="2026-07-30 01:27:23.502259818 +0000 UTC m=+2630.724993636"
Jul 30 01:27:23.518095 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:23.518048    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829" podStartSLOduration=1.518032309 podStartE2EDuration="1.518032309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:27:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:27:23.517272213 +0000 UTC m=+2630.740006058" watchObservedRunningTime="2026-07-30 01:27:23.518032309 +0000 UTC m=+2630.740766127"
Jul 30 01:27:24.491235 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:24.491203    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829"
Jul 30 01:27:28.491422 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:28.491387    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w"
Jul 30 01:27:34.493554 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:34.493508    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm"
Jul 30 01:27:35.498303 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:27:35.498273    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829"
Jul 30 01:28:33.412449 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:28:33.412377    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 01:28:33.416051 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:28:33.416028    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 01:29:31.973749 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:31.973710    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv"]
Jul 30 01:29:32.024000 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.023967    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h"]
Jul 30 01:29:32.024248 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.024222    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h" podUID="f66b1606-c471-4488-849d-996048b6c1d9" containerName="payload-processing" containerID="cri-o://c775289c227059c9cc8fd6f456d6a2253020aebf9a2261c4fcfa95afb8c7095b" gracePeriod=30
Jul 30 01:29:32.033014 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.032827    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc"]
Jul 30 01:29:32.033093 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.033067    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc" podUID="e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9" containerName="payload-pre-processing" containerID="cri-o://f16067b5e40abf9b635a5db533bd8c3fa18ddacd9314e6637add475e6d727f9a" gracePeriod=30
Jul 30 01:29:32.164875 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.164839    2546 generic.go:356] "Generic (PLEG): container finished" podID="e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9" containerID="f16067b5e40abf9b635a5db533bd8c3fa18ddacd9314e6637add475e6d727f9a" exitCode=0
Jul 30 01:29:32.165017 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.164903    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc" event={"ID":"e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9","Type":"ContainerDied","Data":"f16067b5e40abf9b635a5db533bd8c3fa18ddacd9314e6637add475e6d727f9a"}
Jul 30 01:29:32.166620 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.166596    2546 generic.go:356] "Generic (PLEG): container finished" podID="f66b1606-c471-4488-849d-996048b6c1d9" containerID="c775289c227059c9cc8fd6f456d6a2253020aebf9a2261c4fcfa95afb8c7095b" exitCode=0
Jul 30 01:29:32.166753 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.166668    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h" event={"ID":"f66b1606-c471-4488-849d-996048b6c1d9","Type":"ContainerDied","Data":"c775289c227059c9cc8fd6f456d6a2253020aebf9a2261c4fcfa95afb8c7095b"}
Jul 30 01:29:32.166821 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.166801    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv" podUID="0d9148ec-84c7-48af-918d-8abd007f9a8c" containerName="maas-api" containerID="cri-o://e28d1f4b5d2e4369d906325fa8e84ebaac39ed5e8b142d8500ebe6401e8c4e03" gracePeriod=30
Jul 30 01:29:32.322180 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.322047    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h"
Jul 30 01:29:32.324791 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.324768    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc"
Jul 30 01:29:32.386702 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.386665    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97gd5\" (UniqueName: \"kubernetes.io/projected/f66b1606-c471-4488-849d-996048b6c1d9-kube-api-access-97gd5\") pod \"f66b1606-c471-4488-849d-996048b6c1d9\" (UID: \"f66b1606-c471-4488-849d-996048b6c1d9\") "
Jul 30 01:29:32.386878 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.386756    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f66b1606-c471-4488-849d-996048b6c1d9-plugins-config-volume\") pod \"f66b1606-c471-4488-849d-996048b6c1d9\" (UID: \"f66b1606-c471-4488-849d-996048b6c1d9\") "
Jul 30 01:29:32.386964 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.386929    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9-plugins-config-volume\") pod \"e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9\" (UID: \"e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9\") "
Jul 30 01:29:32.387030 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.386997    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2dbz\" (UniqueName: \"kubernetes.io/projected/e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9-kube-api-access-k2dbz\") pod \"e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9\" (UID: \"e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9\") "
Jul 30 01:29:32.387712 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.387442    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9" (UID: "e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:29:32.388158 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.388130    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f66b1606-c471-4488-849d-996048b6c1d9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f66b1606-c471-4488-849d-996048b6c1d9" (UID: "f66b1606-c471-4488-849d-996048b6c1d9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:29:32.390026 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.389989    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9-kube-api-access-k2dbz" (OuterVolumeSpecName: "kube-api-access-k2dbz") pod "e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9" (UID: "e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9"). InnerVolumeSpecName "kube-api-access-k2dbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:29:32.390224 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.390199    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f66b1606-c471-4488-849d-996048b6c1d9-kube-api-access-97gd5" (OuterVolumeSpecName: "kube-api-access-97gd5") pod "f66b1606-c471-4488-849d-996048b6c1d9" (UID: "f66b1606-c471-4488-849d-996048b6c1d9"). InnerVolumeSpecName "kube-api-access-97gd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:29:32.429098 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.429071    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv"
Jul 30 01:29:32.488324 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.488291    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-982gp\" (UniqueName: \"kubernetes.io/projected/0d9148ec-84c7-48af-918d-8abd007f9a8c-kube-api-access-982gp\") pod \"0d9148ec-84c7-48af-918d-8abd007f9a8c\" (UID: \"0d9148ec-84c7-48af-918d-8abd007f9a8c\") "
Jul 30 01:29:32.488531 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.488397    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d9148ec-84c7-48af-918d-8abd007f9a8c-openshift-service-ca\") pod \"0d9148ec-84c7-48af-918d-8abd007f9a8c\" (UID: \"0d9148ec-84c7-48af-918d-8abd007f9a8c\") "
Jul 30 01:29:32.488531 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.488439    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d9148ec-84c7-48af-918d-8abd007f9a8c-maas-api-tls\") pod \"0d9148ec-84c7-48af-918d-8abd007f9a8c\" (UID: \"0d9148ec-84c7-48af-918d-8abd007f9a8c\") "
Jul 30 01:29:32.488728 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.488710    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97gd5\" (UniqueName: \"kubernetes.io/projected/f66b1606-c471-4488-849d-996048b6c1d9-kube-api-access-97gd5\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:29:32.488793 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.488732    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f66b1606-c471-4488-849d-996048b6c1d9-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:29:32.488793 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.488746    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:29:32.488793 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.488759    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2dbz\" (UniqueName: \"kubernetes.io/projected/e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9-kube-api-access-k2dbz\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:29:32.488955 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.488811    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d9148ec-84c7-48af-918d-8abd007f9a8c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0d9148ec-84c7-48af-918d-8abd007f9a8c" (UID: "0d9148ec-84c7-48af-918d-8abd007f9a8c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:29:32.490508 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.490446    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d9148ec-84c7-48af-918d-8abd007f9a8c-kube-api-access-982gp" (OuterVolumeSpecName: "kube-api-access-982gp") pod "0d9148ec-84c7-48af-918d-8abd007f9a8c" (UID: "0d9148ec-84c7-48af-918d-8abd007f9a8c"). InnerVolumeSpecName "kube-api-access-982gp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:29:32.490508 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.490493    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d9148ec-84c7-48af-918d-8abd007f9a8c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0d9148ec-84c7-48af-918d-8abd007f9a8c" (UID: "0d9148ec-84c7-48af-918d-8abd007f9a8c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:29:32.590082 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.590044    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-982gp\" (UniqueName: \"kubernetes.io/projected/0d9148ec-84c7-48af-918d-8abd007f9a8c-kube-api-access-982gp\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:29:32.590082 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.590082    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d9148ec-84c7-48af-918d-8abd007f9a8c-openshift-service-ca\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:29:32.590082 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:32.590097    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d9148ec-84c7-48af-918d-8abd007f9a8c-maas-api-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:29:33.173547 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:33.173500    2546 generic.go:356] "Generic (PLEG): container finished" podID="0d9148ec-84c7-48af-918d-8abd007f9a8c" containerID="e28d1f4b5d2e4369d906325fa8e84ebaac39ed5e8b142d8500ebe6401e8c4e03" exitCode=0
Jul 30 01:29:33.174003 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:33.173572    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv"
Jul 30 01:29:33.174003 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:33.173586    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv" event={"ID":"0d9148ec-84c7-48af-918d-8abd007f9a8c","Type":"ContainerDied","Data":"e28d1f4b5d2e4369d906325fa8e84ebaac39ed5e8b142d8500ebe6401e8c4e03"}
Jul 30 01:29:33.174003 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:33.173668    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv" event={"ID":"0d9148ec-84c7-48af-918d-8abd007f9a8c","Type":"ContainerDied","Data":"869ec035577be3e8243300dd593301f05fab856fca4b3336197aaf5505dea1a4"}
Jul 30 01:29:33.174003 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:33.173702    2546 scope.go:117] "RemoveContainer" containerID="e28d1f4b5d2e4369d906325fa8e84ebaac39ed5e8b142d8500ebe6401e8c4e03"
Jul 30 01:29:33.175066 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:33.174951    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc"
Jul 30 01:29:33.175066 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:33.174978    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc" event={"ID":"e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9","Type":"ContainerDied","Data":"d01192fc250303fc6ea4248e5347121e76379933d500e74aef5d681331b4a26c"}
Jul 30 01:29:33.176472 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:33.176441    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h" event={"ID":"f66b1606-c471-4488-849d-996048b6c1d9","Type":"ContainerDied","Data":"85a3cc9d96df5c3faee7d450ce1aae9c632ee3061c240713a54187b7889cfbf0"}
Jul 30 01:29:33.176603 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:33.176490    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h"
Jul 30 01:29:33.183117 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:33.183071    2546 scope.go:117] "RemoveContainer" containerID="e28d1f4b5d2e4369d906325fa8e84ebaac39ed5e8b142d8500ebe6401e8c4e03"
Jul 30 01:29:33.183351 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:29:33.183332    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e28d1f4b5d2e4369d906325fa8e84ebaac39ed5e8b142d8500ebe6401e8c4e03\": container with ID starting with e28d1f4b5d2e4369d906325fa8e84ebaac39ed5e8b142d8500ebe6401e8c4e03 not found: ID does not exist" containerID="e28d1f4b5d2e4369d906325fa8e84ebaac39ed5e8b142d8500ebe6401e8c4e03"
Jul 30 01:29:33.183429 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:33.183357    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e28d1f4b5d2e4369d906325fa8e84ebaac39ed5e8b142d8500ebe6401e8c4e03"} err="failed to get container status \"e28d1f4b5d2e4369d906325fa8e84ebaac39ed5e8b142d8500ebe6401e8c4e03\": rpc error: code = NotFound desc = could not find container \"e28d1f4b5d2e4369d906325fa8e84ebaac39ed5e8b142d8500ebe6401e8c4e03\": container with ID starting with e28d1f4b5d2e4369d906325fa8e84ebaac39ed5e8b142d8500ebe6401e8c4e03 not found: ID does not exist"
Jul 30 01:29:33.183429 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:33.183373    2546 scope.go:117] "RemoveContainer" containerID="f16067b5e40abf9b635a5db533bd8c3fa18ddacd9314e6637add475e6d727f9a"
Jul 30 01:29:33.193299 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:33.193267    2546 scope.go:117] "RemoveContainer" containerID="c775289c227059c9cc8fd6f456d6a2253020aebf9a2261c4fcfa95afb8c7095b"
Jul 30 01:29:33.202410 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:33.202390    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv"]
Jul 30 01:29:33.205940 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:33.205917    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6ce027-7cb5c6c996-v76rv"]
Jul 30 01:29:33.215713 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:33.215694    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc"]
Jul 30 01:29:33.220324 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:33.220303    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6ce027-78bffbb456-vpkhc"]
Jul 30 01:29:33.229395 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:33.229369    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h"]
Jul 30 01:29:33.235369 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:33.235340    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6ce027-854b75c788-5qt2h"]
Jul 30 01:29:33.358448 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:33.358371    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d9148ec-84c7-48af-918d-8abd007f9a8c" path="/var/lib/kubelet/pods/0d9148ec-84c7-48af-918d-8abd007f9a8c/volumes"
Jul 30 01:29:33.358781 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:33.358766    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9" path="/var/lib/kubelet/pods/e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9/volumes"
Jul 30 01:29:33.359083 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:33.359071    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f66b1606-c471-4488-849d-996048b6c1d9" path="/var/lib/kubelet/pods/f66b1606-c471-4488-849d-996048b6c1d9/volumes"
Jul 30 01:29:49.396813 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.396778    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w"]
Jul 30 01:29:49.397412 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.396997    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w" podUID="7b9d4e64-064f-4815-8315-c20c3de7a494" containerName="maas-api" containerID="cri-o://8f6794a70a87e15c1c5637c235484be691c25af1ff14380916f30e097a06f409" gracePeriod=30
Jul 30 01:29:49.459656 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.459616    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829"]
Jul 30 01:29:49.459930 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.459889    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829" podUID="40a484b8-6192-444f-8703-97ad2b28938f" containerName="payload-processing" containerID="cri-o://cc6ad6cf1ae078167165d5806cbb015de9e09453cf25e73a8ec6b7a41fe1fc0b" gracePeriod=30
Jul 30 01:29:49.468299 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.468269    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm"]
Jul 30 01:29:49.468577 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.468553    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm" podUID="ebdac8f0-a89f-4814-b773-9eff0b728e3a" containerName="payload-pre-processing" containerID="cri-o://c71dde941788470b2e75524c2ce99399aa6b3d46b88e6bf1fa7b78131c59dd01" gracePeriod=30
Jul 30 01:29:49.763862 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.763841    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829"
Jul 30 01:29:49.771100 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.771075    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm"
Jul 30 01:29:49.780784 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.780765    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w"
Jul 30 01:29:49.841093 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.841066    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbw82\" (UniqueName: \"kubernetes.io/projected/ebdac8f0-a89f-4814-b773-9eff0b728e3a-kube-api-access-fbw82\") pod \"ebdac8f0-a89f-4814-b773-9eff0b728e3a\" (UID: \"ebdac8f0-a89f-4814-b773-9eff0b728e3a\") "
Jul 30 01:29:49.841254 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.841110    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebdac8f0-a89f-4814-b773-9eff0b728e3a-plugins-config-volume\") pod \"ebdac8f0-a89f-4814-b773-9eff0b728e3a\" (UID: \"ebdac8f0-a89f-4814-b773-9eff0b728e3a\") "
Jul 30 01:29:49.841254 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.841179    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ztnr4\" (UniqueName: \"kubernetes.io/projected/40a484b8-6192-444f-8703-97ad2b28938f-kube-api-access-ztnr4\") pod \"40a484b8-6192-444f-8703-97ad2b28938f\" (UID: \"40a484b8-6192-444f-8703-97ad2b28938f\") "
Jul 30 01:29:49.841356 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.841262    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40a484b8-6192-444f-8703-97ad2b28938f-plugins-config-volume\") pod \"40a484b8-6192-444f-8703-97ad2b28938f\" (UID: \"40a484b8-6192-444f-8703-97ad2b28938f\") "
Jul 30 01:29:49.841660 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.841634    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebdac8f0-a89f-4814-b773-9eff0b728e3a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ebdac8f0-a89f-4814-b773-9eff0b728e3a" (UID: "ebdac8f0-a89f-4814-b773-9eff0b728e3a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:29:49.841660 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.841645    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40a484b8-6192-444f-8703-97ad2b28938f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "40a484b8-6192-444f-8703-97ad2b28938f" (UID: "40a484b8-6192-444f-8703-97ad2b28938f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:29:49.843220 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.843202    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40a484b8-6192-444f-8703-97ad2b28938f-kube-api-access-ztnr4" (OuterVolumeSpecName: "kube-api-access-ztnr4") pod "40a484b8-6192-444f-8703-97ad2b28938f" (UID: "40a484b8-6192-444f-8703-97ad2b28938f"). InnerVolumeSpecName "kube-api-access-ztnr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:29:49.843281 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.843234    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebdac8f0-a89f-4814-b773-9eff0b728e3a-kube-api-access-fbw82" (OuterVolumeSpecName: "kube-api-access-fbw82") pod "ebdac8f0-a89f-4814-b773-9eff0b728e3a" (UID: "ebdac8f0-a89f-4814-b773-9eff0b728e3a"). InnerVolumeSpecName "kube-api-access-fbw82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:29:49.942315 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.942236    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b9d4e64-064f-4815-8315-c20c3de7a494-maas-api-tls\") pod \"7b9d4e64-064f-4815-8315-c20c3de7a494\" (UID: \"7b9d4e64-064f-4815-8315-c20c3de7a494\") "
Jul 30 01:29:49.942315 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.942308    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xj4gj\" (UniqueName: \"kubernetes.io/projected/7b9d4e64-064f-4815-8315-c20c3de7a494-kube-api-access-xj4gj\") pod \"7b9d4e64-064f-4815-8315-c20c3de7a494\" (UID: \"7b9d4e64-064f-4815-8315-c20c3de7a494\") "
Jul 30 01:29:49.942563 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.942338    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b9d4e64-064f-4815-8315-c20c3de7a494-openshift-service-ca\") pod \"7b9d4e64-064f-4815-8315-c20c3de7a494\" (UID: \"7b9d4e64-064f-4815-8315-c20c3de7a494\") "
Jul 30 01:29:49.942563 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.942494    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztnr4\" (UniqueName: \"kubernetes.io/projected/40a484b8-6192-444f-8703-97ad2b28938f-kube-api-access-ztnr4\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:29:49.942563 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.942506    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40a484b8-6192-444f-8703-97ad2b28938f-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:29:49.942563 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.942516    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbw82\" (UniqueName: \"kubernetes.io/projected/ebdac8f0-a89f-4814-b773-9eff0b728e3a-kube-api-access-fbw82\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:29:49.942563 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.942560    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebdac8f0-a89f-4814-b773-9eff0b728e3a-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:29:49.942806 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.942779    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b9d4e64-064f-4815-8315-c20c3de7a494-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7b9d4e64-064f-4815-8315-c20c3de7a494" (UID: "7b9d4e64-064f-4815-8315-c20c3de7a494"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:29:49.944412 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.944391    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b9d4e64-064f-4815-8315-c20c3de7a494-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7b9d4e64-064f-4815-8315-c20c3de7a494" (UID: "7b9d4e64-064f-4815-8315-c20c3de7a494"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:29:49.944485 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:49.944431    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b9d4e64-064f-4815-8315-c20c3de7a494-kube-api-access-xj4gj" (OuterVolumeSpecName: "kube-api-access-xj4gj") pod "7b9d4e64-064f-4815-8315-c20c3de7a494" (UID: "7b9d4e64-064f-4815-8315-c20c3de7a494"). InnerVolumeSpecName "kube-api-access-xj4gj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:29:50.043497 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.043463    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b9d4e64-064f-4815-8315-c20c3de7a494-maas-api-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:29:50.043497 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.043495    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xj4gj\" (UniqueName: \"kubernetes.io/projected/7b9d4e64-064f-4815-8315-c20c3de7a494-kube-api-access-xj4gj\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:29:50.043497 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.043504    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b9d4e64-064f-4815-8315-c20c3de7a494-openshift-service-ca\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:29:50.264634 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.264585    2546 generic.go:356] "Generic (PLEG): container finished" podID="ebdac8f0-a89f-4814-b773-9eff0b728e3a" containerID="c71dde941788470b2e75524c2ce99399aa6b3d46b88e6bf1fa7b78131c59dd01" exitCode=0
Jul 30 01:29:50.264634 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.264642    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm"
Jul 30 01:29:50.264895 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.264666    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm" event={"ID":"ebdac8f0-a89f-4814-b773-9eff0b728e3a","Type":"ContainerDied","Data":"c71dde941788470b2e75524c2ce99399aa6b3d46b88e6bf1fa7b78131c59dd01"}
Jul 30 01:29:50.264895 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.264756    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm" event={"ID":"ebdac8f0-a89f-4814-b773-9eff0b728e3a","Type":"ContainerDied","Data":"5913418bdbb04a25c94fa320f9526a0d48015b96ca4639f9a0fa6502ddaec257"}
Jul 30 01:29:50.264895 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.264789    2546 scope.go:117] "RemoveContainer" containerID="c71dde941788470b2e75524c2ce99399aa6b3d46b88e6bf1fa7b78131c59dd01"
Jul 30 01:29:50.265955 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.265929    2546 generic.go:356] "Generic (PLEG): container finished" podID="40a484b8-6192-444f-8703-97ad2b28938f" containerID="cc6ad6cf1ae078167165d5806cbb015de9e09453cf25e73a8ec6b7a41fe1fc0b" exitCode=0
Jul 30 01:29:50.266075 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.265960    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829" event={"ID":"40a484b8-6192-444f-8703-97ad2b28938f","Type":"ContainerDied","Data":"cc6ad6cf1ae078167165d5806cbb015de9e09453cf25e73a8ec6b7a41fe1fc0b"}
Jul 30 01:29:50.266075 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.266005    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829"
Jul 30 01:29:50.266075 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.266005    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829" event={"ID":"40a484b8-6192-444f-8703-97ad2b28938f","Type":"ContainerDied","Data":"860b8af55915c8e744ca4b0bd9676bf6e274195c35786372caa8e339f33f12db"}
Jul 30 01:29:50.267371 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.267346    2546 generic.go:356] "Generic (PLEG): container finished" podID="7b9d4e64-064f-4815-8315-c20c3de7a494" containerID="8f6794a70a87e15c1c5637c235484be691c25af1ff14380916f30e097a06f409" exitCode=0
Jul 30 01:29:50.267473 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.267396    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w"
Jul 30 01:29:50.267473 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.267436    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w" event={"ID":"7b9d4e64-064f-4815-8315-c20c3de7a494","Type":"ContainerDied","Data":"8f6794a70a87e15c1c5637c235484be691c25af1ff14380916f30e097a06f409"}
Jul 30 01:29:50.267606 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.267483    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w" event={"ID":"7b9d4e64-064f-4815-8315-c20c3de7a494","Type":"ContainerDied","Data":"ee164f89ba19dc54d4c91e7341f9d1676640bb8b350a495d1320568c78b8487c"}
Jul 30 01:29:50.276335 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.276319    2546 scope.go:117] "RemoveContainer" containerID="c71dde941788470b2e75524c2ce99399aa6b3d46b88e6bf1fa7b78131c59dd01"
Jul 30 01:29:50.276629 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:29:50.276605    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c71dde941788470b2e75524c2ce99399aa6b3d46b88e6bf1fa7b78131c59dd01\": container with ID starting with c71dde941788470b2e75524c2ce99399aa6b3d46b88e6bf1fa7b78131c59dd01 not found: ID does not exist" containerID="c71dde941788470b2e75524c2ce99399aa6b3d46b88e6bf1fa7b78131c59dd01"
Jul 30 01:29:50.276735 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.276633    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c71dde941788470b2e75524c2ce99399aa6b3d46b88e6bf1fa7b78131c59dd01"} err="failed to get container status \"c71dde941788470b2e75524c2ce99399aa6b3d46b88e6bf1fa7b78131c59dd01\": rpc error: code = NotFound desc = could not find container \"c71dde941788470b2e75524c2ce99399aa6b3d46b88e6bf1fa7b78131c59dd01\": container with ID starting with c71dde941788470b2e75524c2ce99399aa6b3d46b88e6bf1fa7b78131c59dd01 not found: ID does not exist"
Jul 30 01:29:50.276735 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.276655    2546 scope.go:117] "RemoveContainer" containerID="cc6ad6cf1ae078167165d5806cbb015de9e09453cf25e73a8ec6b7a41fe1fc0b"
Jul 30 01:29:50.284372 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.284356    2546 scope.go:117] "RemoveContainer" containerID="cc6ad6cf1ae078167165d5806cbb015de9e09453cf25e73a8ec6b7a41fe1fc0b"
Jul 30 01:29:50.284668 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:29:50.284647    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc6ad6cf1ae078167165d5806cbb015de9e09453cf25e73a8ec6b7a41fe1fc0b\": container with ID starting with cc6ad6cf1ae078167165d5806cbb015de9e09453cf25e73a8ec6b7a41fe1fc0b not found: ID does not exist" containerID="cc6ad6cf1ae078167165d5806cbb015de9e09453cf25e73a8ec6b7a41fe1fc0b"
Jul 30 01:29:50.284768 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.284672    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc6ad6cf1ae078167165d5806cbb015de9e09453cf25e73a8ec6b7a41fe1fc0b"} err="failed to get container status \"cc6ad6cf1ae078167165d5806cbb015de9e09453cf25e73a8ec6b7a41fe1fc0b\": rpc error: code = NotFound desc = could not find container \"cc6ad6cf1ae078167165d5806cbb015de9e09453cf25e73a8ec6b7a41fe1fc0b\": container with ID starting with cc6ad6cf1ae078167165d5806cbb015de9e09453cf25e73a8ec6b7a41fe1fc0b not found: ID does not exist"
Jul 30 01:29:50.284768 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.284687    2546 scope.go:117] "RemoveContainer" containerID="8f6794a70a87e15c1c5637c235484be691c25af1ff14380916f30e097a06f409"
Jul 30 01:29:50.291355 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.291329    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm"]
Jul 30 01:29:50.294172 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.294147    2546 scope.go:117] "RemoveContainer" containerID="8f6794a70a87e15c1c5637c235484be691c25af1ff14380916f30e097a06f409"
Jul 30 01:29:50.294433 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:29:50.294406    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f6794a70a87e15c1c5637c235484be691c25af1ff14380916f30e097a06f409\": container with ID starting with 8f6794a70a87e15c1c5637c235484be691c25af1ff14380916f30e097a06f409 not found: ID does not exist" containerID="8f6794a70a87e15c1c5637c235484be691c25af1ff14380916f30e097a06f409"
Jul 30 01:29:50.294499 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.294432    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f6794a70a87e15c1c5637c235484be691c25af1ff14380916f30e097a06f409"} err="failed to get container status \"8f6794a70a87e15c1c5637c235484be691c25af1ff14380916f30e097a06f409\": rpc error: code = NotFound desc = could not find container \"8f6794a70a87e15c1c5637c235484be691c25af1ff14380916f30e097a06f409\": container with ID starting with 8f6794a70a87e15c1c5637c235484be691c25af1ff14380916f30e097a06f409 not found: ID does not exist"
Jul 30 01:29:50.295892 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.295874    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-b2ddd3-fbc6597b-sw2nm"]
Jul 30 01:29:50.306730 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.306677    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w"]
Jul 30 01:29:50.308726 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.308703    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b2ddd3-7df69559f9-t7q6w"]
Jul 30 01:29:50.318656 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.318632    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829"]
Jul 30 01:29:50.322688 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:50.322666    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-b2ddd3-6646ccb4d6-bz829"]
Jul 30 01:29:51.358499 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:51.358454    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40a484b8-6192-444f-8703-97ad2b28938f" path="/var/lib/kubelet/pods/40a484b8-6192-444f-8703-97ad2b28938f/volumes"
Jul 30 01:29:51.359070 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:51.359049    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b9d4e64-064f-4815-8315-c20c3de7a494" path="/var/lib/kubelet/pods/7b9d4e64-064f-4815-8315-c20c3de7a494/volumes"
Jul 30 01:29:51.359443 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:29:51.359424    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebdac8f0-a89f-4814-b773-9eff0b728e3a" path="/var/lib/kubelet/pods/ebdac8f0-a89f-4814-b773-9eff0b728e3a/volumes"
Jul 30 01:30:00.934617 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:00.934513    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx"]
Jul 30 01:30:00.935146 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:00.935117    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9" containerName="payload-pre-processing"
Jul 30 01:30:00.935146 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:00.935137    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9" containerName="payload-pre-processing"
Jul 30 01:30:00.935307 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:00.935169    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d9148ec-84c7-48af-918d-8abd007f9a8c" containerName="maas-api"
Jul 30 01:30:00.935307 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:00.935177    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d9148ec-84c7-48af-918d-8abd007f9a8c" containerName="maas-api"
Jul 30 01:30:00.935307 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:00.935191    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f66b1606-c471-4488-849d-996048b6c1d9" containerName="payload-processing"
Jul 30 01:30:00.935307 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:00.935200    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="f66b1606-c471-4488-849d-996048b6c1d9" containerName="payload-processing"
Jul 30 01:30:00.935307 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:00.935209    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b9d4e64-064f-4815-8315-c20c3de7a494" containerName="maas-api"
Jul 30 01:30:00.935307 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:00.935217    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b9d4e64-064f-4815-8315-c20c3de7a494" containerName="maas-api"
Jul 30 01:30:00.935307 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:00.935233    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebdac8f0-a89f-4814-b773-9eff0b728e3a" containerName="payload-pre-processing"
Jul 30 01:30:00.935307 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:00.935241    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebdac8f0-a89f-4814-b773-9eff0b728e3a" containerName="payload-pre-processing"
Jul 30 01:30:00.935307 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:00.935253    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40a484b8-6192-444f-8703-97ad2b28938f" containerName="payload-processing"
Jul 30 01:30:00.935307 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:00.935262    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="40a484b8-6192-444f-8703-97ad2b28938f" containerName="payload-processing"
Jul 30 01:30:00.935803 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:00.935359    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d9148ec-84c7-48af-918d-8abd007f9a8c" containerName="maas-api"
Jul 30 01:30:00.935803 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:00.935374    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebdac8f0-a89f-4814-b773-9eff0b728e3a" containerName="payload-pre-processing"
Jul 30 01:30:00.935803 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:00.935386    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b9d4e64-064f-4815-8315-c20c3de7a494" containerName="maas-api"
Jul 30 01:30:00.935803 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:00.935397    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="f66b1606-c471-4488-849d-996048b6c1d9" containerName="payload-processing"
Jul 30 01:30:00.935803 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:00.935408    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="40a484b8-6192-444f-8703-97ad2b28938f" containerName="payload-processing"
Jul 30 01:30:00.935803 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:00.935416    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1dc6e10-b79e-47ff-a07d-b7cc4219c1b9" containerName="payload-pre-processing"
Jul 30 01:30:00.940398 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:00.940375    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx"
Jul 30 01:30:00.943138 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:00.943115    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-5ca190-serving-cert\""
Jul 30 01:30:00.951985 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:00.951955    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx"]
Jul 30 01:30:01.039105 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.039067    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7651aa76-d4bb-4f06-96ae-afe66836112c-openshift-service-ca\") pod \"maas-api-e2e-api-a-5ca190-7657548f9f-84dsx\" (UID: \"7651aa76-d4bb-4f06-96ae-afe66836112c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx"
Jul 30 01:30:01.039105 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.039101    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fcfj\" (UniqueName: \"kubernetes.io/projected/7651aa76-d4bb-4f06-96ae-afe66836112c-kube-api-access-6fcfj\") pod \"maas-api-e2e-api-a-5ca190-7657548f9f-84dsx\" (UID: \"7651aa76-d4bb-4f06-96ae-afe66836112c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx"
Jul 30 01:30:01.039330 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.039144    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7651aa76-d4bb-4f06-96ae-afe66836112c-maas-api-tls\") pod \"maas-api-e2e-api-a-5ca190-7657548f9f-84dsx\" (UID: \"7651aa76-d4bb-4f06-96ae-afe66836112c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx"
Jul 30 01:30:01.139662 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.139628    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7651aa76-d4bb-4f06-96ae-afe66836112c-maas-api-tls\") pod \"maas-api-e2e-api-a-5ca190-7657548f9f-84dsx\" (UID: \"7651aa76-d4bb-4f06-96ae-afe66836112c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx"
Jul 30 01:30:01.139845 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.139737    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7651aa76-d4bb-4f06-96ae-afe66836112c-openshift-service-ca\") pod \"maas-api-e2e-api-a-5ca190-7657548f9f-84dsx\" (UID: \"7651aa76-d4bb-4f06-96ae-afe66836112c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx"
Jul 30 01:30:01.139845 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.139763    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fcfj\" (UniqueName: \"kubernetes.io/projected/7651aa76-d4bb-4f06-96ae-afe66836112c-kube-api-access-6fcfj\") pod \"maas-api-e2e-api-a-5ca190-7657548f9f-84dsx\" (UID: \"7651aa76-d4bb-4f06-96ae-afe66836112c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx"
Jul 30 01:30:01.139845 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:30:01.139786    2546 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-serving-cert: secret "maas-api-e2e-api-a-5ca190-serving-cert" not found
Jul 30 01:30:01.139979 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:30:01.139848    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7651aa76-d4bb-4f06-96ae-afe66836112c-maas-api-tls podName:7651aa76-d4bb-4f06-96ae-afe66836112c nodeName:}" failed. No retries permitted until 2026-07-30 01:30:01.639828681 +0000 UTC m=+2788.862562492 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7651aa76-d4bb-4f06-96ae-afe66836112c-maas-api-tls") pod "maas-api-e2e-api-a-5ca190-7657548f9f-84dsx" (UID: "7651aa76-d4bb-4f06-96ae-afe66836112c") : secret "maas-api-e2e-api-a-5ca190-serving-cert" not found
Jul 30 01:30:01.140563 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.140506    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7651aa76-d4bb-4f06-96ae-afe66836112c-openshift-service-ca\") pod \"maas-api-e2e-api-a-5ca190-7657548f9f-84dsx\" (UID: \"7651aa76-d4bb-4f06-96ae-afe66836112c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx"
Jul 30 01:30:01.149923 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.149899    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fcfj\" (UniqueName: \"kubernetes.io/projected/7651aa76-d4bb-4f06-96ae-afe66836112c-kube-api-access-6fcfj\") pod \"maas-api-e2e-api-a-5ca190-7657548f9f-84dsx\" (UID: \"7651aa76-d4bb-4f06-96ae-afe66836112c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx"
Jul 30 01:30:01.608275 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.605288    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t"]
Jul 30 01:30:01.610611 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.610584    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t"
Jul 30 01:30:01.613347 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.613322    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-5ca190-dockercfg-st5qc\""
Jul 30 01:30:01.613500 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.613484    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-5ca190\""
Jul 30 01:30:01.617499 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.617474    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t"]
Jul 30 01:30:01.644807 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.644764    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7651aa76-d4bb-4f06-96ae-afe66836112c-maas-api-tls\") pod \"maas-api-e2e-api-a-5ca190-7657548f9f-84dsx\" (UID: \"7651aa76-d4bb-4f06-96ae-afe66836112c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx"
Jul 30 01:30:01.647323 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.647256    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7651aa76-d4bb-4f06-96ae-afe66836112c-maas-api-tls\") pod \"maas-api-e2e-api-a-5ca190-7657548f9f-84dsx\" (UID: \"7651aa76-d4bb-4f06-96ae-afe66836112c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx"
Jul 30 01:30:01.666353 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.666319    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p"]
Jul 30 01:30:01.669831 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.669815    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p"
Jul 30 01:30:01.680659 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.680635    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p"]
Jul 30 01:30:01.746339 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.746304    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15828e66-2547-4c5e-b94f-ff2ee1b75d86-plugins-config-volume\") pod \"payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t\" (UID: \"15828e66-2547-4c5e-b94f-ff2ee1b75d86\") " pod="openshift-ingress/payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t"
Jul 30 01:30:01.746516 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.746443    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t94rv\" (UniqueName: \"kubernetes.io/projected/15828e66-2547-4c5e-b94f-ff2ee1b75d86-kube-api-access-t94rv\") pod \"payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t\" (UID: \"15828e66-2547-4c5e-b94f-ff2ee1b75d86\") " pod="openshift-ingress/payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t"
Jul 30 01:30:01.846984 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.846946    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3046f61-9c6c-41eb-87e3-3046708f4c7d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p\" (UID: \"a3046f61-9c6c-41eb-87e3-3046708f4c7d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p"
Jul 30 01:30:01.847158 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.847009    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t94rv\" (UniqueName: \"kubernetes.io/projected/15828e66-2547-4c5e-b94f-ff2ee1b75d86-kube-api-access-t94rv\") pod \"payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t\" (UID: \"15828e66-2547-4c5e-b94f-ff2ee1b75d86\") " pod="openshift-ingress/payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t"
Jul 30 01:30:01.847158 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.847064    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15828e66-2547-4c5e-b94f-ff2ee1b75d86-plugins-config-volume\") pod \"payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t\" (UID: \"15828e66-2547-4c5e-b94f-ff2ee1b75d86\") " pod="openshift-ingress/payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t"
Jul 30 01:30:01.847158 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.847118    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw5fs\" (UniqueName: \"kubernetes.io/projected/a3046f61-9c6c-41eb-87e3-3046708f4c7d-kube-api-access-fw5fs\") pod \"payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p\" (UID: \"a3046f61-9c6c-41eb-87e3-3046708f4c7d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p"
Jul 30 01:30:01.847816 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.847797    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15828e66-2547-4c5e-b94f-ff2ee1b75d86-plugins-config-volume\") pod \"payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t\" (UID: \"15828e66-2547-4c5e-b94f-ff2ee1b75d86\") " pod="openshift-ingress/payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t"
Jul 30 01:30:01.852807 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.852782    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx"
Jul 30 01:30:01.855756 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.855730    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t94rv\" (UniqueName: \"kubernetes.io/projected/15828e66-2547-4c5e-b94f-ff2ee1b75d86-kube-api-access-t94rv\") pod \"payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t\" (UID: \"15828e66-2547-4c5e-b94f-ff2ee1b75d86\") " pod="openshift-ingress/payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t"
Jul 30 01:30:01.925192 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.925147    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t"
Jul 30 01:30:01.949740 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.949702    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw5fs\" (UniqueName: \"kubernetes.io/projected/a3046f61-9c6c-41eb-87e3-3046708f4c7d-kube-api-access-fw5fs\") pod \"payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p\" (UID: \"a3046f61-9c6c-41eb-87e3-3046708f4c7d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p"
Jul 30 01:30:01.950179 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.949774    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3046f61-9c6c-41eb-87e3-3046708f4c7d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p\" (UID: \"a3046f61-9c6c-41eb-87e3-3046708f4c7d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p"
Jul 30 01:30:01.950701 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.950667    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3046f61-9c6c-41eb-87e3-3046708f4c7d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p\" (UID: \"a3046f61-9c6c-41eb-87e3-3046708f4c7d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p"
Jul 30 01:30:01.959215 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.959183    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw5fs\" (UniqueName: \"kubernetes.io/projected/a3046f61-9c6c-41eb-87e3-3046708f4c7d-kube-api-access-fw5fs\") pod \"payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p\" (UID: \"a3046f61-9c6c-41eb-87e3-3046708f4c7d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p"
Jul 30 01:30:01.981481 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:01.981447    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p"
Jul 30 01:30:02.010055 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:02.010023    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx"]
Jul 30 01:30:02.011744 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:30:02.011636    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7651aa76_d4bb_4f06_96ae_afe66836112c.slice/crio-3fa745280b89352c3bb17e471216819269af61e607e15622b0ec03c20d4ba645 WatchSource:0}: Error finding container 3fa745280b89352c3bb17e471216819269af61e607e15622b0ec03c20d4ba645: Status 404 returned error can't find the container with id 3fa745280b89352c3bb17e471216819269af61e607e15622b0ec03c20d4ba645
Jul 30 01:30:02.013601 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:02.013573    2546 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 01:30:02.092101 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:02.092076    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t"]
Jul 30 01:30:02.095038 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:30:02.094908    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15828e66_2547_4c5e_b94f_ff2ee1b75d86.slice/crio-0a9a61b529ac6b9575529375b380873608d37a00e91a228b784723407e8c3de6 WatchSource:0}: Error finding container 0a9a61b529ac6b9575529375b380873608d37a00e91a228b784723407e8c3de6: Status 404 returned error can't find the container with id 0a9a61b529ac6b9575529375b380873608d37a00e91a228b784723407e8c3de6
Jul 30 01:30:02.155343 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:02.155318    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p"]
Jul 30 01:30:02.157458 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:30:02.157432    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3046f61_9c6c_41eb_87e3_3046708f4c7d.slice/crio-d698b03a9f34116b271fac29470ce0a730be5fadbdd8a7bdca8ab3e7c76c78e4 WatchSource:0}: Error finding container d698b03a9f34116b271fac29470ce0a730be5fadbdd8a7bdca8ab3e7c76c78e4: Status 404 returned error can't find the container with id d698b03a9f34116b271fac29470ce0a730be5fadbdd8a7bdca8ab3e7c76c78e4
Jul 30 01:30:02.334912 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:02.334880    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t" event={"ID":"15828e66-2547-4c5e-b94f-ff2ee1b75d86","Type":"ContainerStarted","Data":"76e1c71948c3eb275b8c8742d1d911c0880963c76f9ab416925ef1b4edddce4e"}
Jul 30 01:30:02.335083 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:02.334935    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t" event={"ID":"15828e66-2547-4c5e-b94f-ff2ee1b75d86","Type":"ContainerStarted","Data":"0a9a61b529ac6b9575529375b380873608d37a00e91a228b784723407e8c3de6"}
Jul 30 01:30:02.335083 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:02.334967    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t"
Jul 30 01:30:02.336451 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:02.336427    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p" event={"ID":"a3046f61-9c6c-41eb-87e3-3046708f4c7d","Type":"ContainerStarted","Data":"2187ad56dcf512cf4a443edbd0b11db45a3bfa6034a4810a8729d8baa9daceda"}
Jul 30 01:30:02.336593 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:02.336473    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p" event={"ID":"a3046f61-9c6c-41eb-87e3-3046708f4c7d","Type":"ContainerStarted","Data":"d698b03a9f34116b271fac29470ce0a730be5fadbdd8a7bdca8ab3e7c76c78e4"}
Jul 30 01:30:02.336593 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:02.336560    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p"
Jul 30 01:30:02.337819 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:02.337797    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx" event={"ID":"7651aa76-d4bb-4f06-96ae-afe66836112c","Type":"ContainerStarted","Data":"3fa745280b89352c3bb17e471216819269af61e607e15622b0ec03c20d4ba645"}
Jul 30 01:30:02.350214 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:02.350172    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t" podStartSLOduration=1.3501606480000001 podStartE2EDuration="1.350160648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:30:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:30:02.348864975 +0000 UTC m=+2789.571598804" watchObservedRunningTime="2026-07-30 01:30:02.350160648 +0000 UTC m=+2789.572894465"
Jul 30 01:30:02.364740 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:02.364689    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p" podStartSLOduration=1.364675252 podStartE2EDuration="1.364675252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:30:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:30:02.362753701 +0000 UTC m=+2789.585487518" watchObservedRunningTime="2026-07-30 01:30:02.364675252 +0000 UTC m=+2789.587409069"
Jul 30 01:30:03.344959 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:03.344912    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx" event={"ID":"7651aa76-d4bb-4f06-96ae-afe66836112c","Type":"ContainerStarted","Data":"f5c8a888362b1353ba773e8571fde70a65ca2ab9b7c8973082b5c8c010ebf5cb"}
Jul 30 01:30:03.345467 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:03.345081    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx"
Jul 30 01:30:03.362374 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:03.362314    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx" podStartSLOduration=2.760090609 podStartE2EDuration="3.362298604s" totalImagesPullingTime="602.207995ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:30:03.360203478 +0000 UTC m=+2790.582937295" watchObservedRunningTime="2026-07-30 01:30:03.362298604 +0000 UTC m=+2790.585032425"
Jul 30 01:30:09.359363 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:09.359335    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx"
Jul 30 01:30:13.347509 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:13.347471    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t"
Jul 30 01:30:13.347907 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:13.347654    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p"
Jul 30 01:30:29.553994 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:29.553956    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf"]
Jul 30 01:30:29.557763 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:29.557743    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf"
Jul 30 01:30:29.560783 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:29.560760    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-b11aa3-serving-cert\""
Jul 30 01:30:29.571379 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:29.571357    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf"]
Jul 30 01:30:29.681838 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:29.681782    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5085ae9b-40a4-4eb1-92e7-ca2bd1292272-maas-api-tls\") pod \"maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf\" (UID: \"5085ae9b-40a4-4eb1-92e7-ca2bd1292272\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf"
Jul 30 01:30:29.681838 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:29.681845    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5085ae9b-40a4-4eb1-92e7-ca2bd1292272-openshift-service-ca\") pod \"maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf\" (UID: \"5085ae9b-40a4-4eb1-92e7-ca2bd1292272\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf"
Jul 30 01:30:29.682105 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:29.681918    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxmrv\" (UniqueName: \"kubernetes.io/projected/5085ae9b-40a4-4eb1-92e7-ca2bd1292272-kube-api-access-qxmrv\") pod \"maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf\" (UID: \"5085ae9b-40a4-4eb1-92e7-ca2bd1292272\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf"
Jul 30 01:30:29.783100 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:29.783062    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5085ae9b-40a4-4eb1-92e7-ca2bd1292272-maas-api-tls\") pod \"maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf\" (UID: \"5085ae9b-40a4-4eb1-92e7-ca2bd1292272\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf"
Jul 30 01:30:29.783100 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:29.783103    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5085ae9b-40a4-4eb1-92e7-ca2bd1292272-openshift-service-ca\") pod \"maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf\" (UID: \"5085ae9b-40a4-4eb1-92e7-ca2bd1292272\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf"
Jul 30 01:30:29.783314 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:29.783149    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxmrv\" (UniqueName: \"kubernetes.io/projected/5085ae9b-40a4-4eb1-92e7-ca2bd1292272-kube-api-access-qxmrv\") pod \"maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf\" (UID: \"5085ae9b-40a4-4eb1-92e7-ca2bd1292272\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf"
Jul 30 01:30:29.783314 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:30:29.783211    2546 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-serving-cert: secret "maas-api-e2e-api-b-b11aa3-serving-cert" not found
Jul 30 01:30:29.783314 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:30:29.783305    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5085ae9b-40a4-4eb1-92e7-ca2bd1292272-maas-api-tls podName:5085ae9b-40a4-4eb1-92e7-ca2bd1292272 nodeName:}" failed. No retries permitted until 2026-07-30 01:30:30.283288309 +0000 UTC m=+2817.506022105 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5085ae9b-40a4-4eb1-92e7-ca2bd1292272-maas-api-tls") pod "maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf" (UID: "5085ae9b-40a4-4eb1-92e7-ca2bd1292272") : secret "maas-api-e2e-api-b-b11aa3-serving-cert" not found
Jul 30 01:30:29.783779 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:29.783762    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5085ae9b-40a4-4eb1-92e7-ca2bd1292272-openshift-service-ca\") pod \"maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf\" (UID: \"5085ae9b-40a4-4eb1-92e7-ca2bd1292272\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf"
Jul 30 01:30:29.794550 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:29.794493    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxmrv\" (UniqueName: \"kubernetes.io/projected/5085ae9b-40a4-4eb1-92e7-ca2bd1292272-kube-api-access-qxmrv\") pod \"maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf\" (UID: \"5085ae9b-40a4-4eb1-92e7-ca2bd1292272\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf"
Jul 30 01:30:30.287758 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.287721    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5085ae9b-40a4-4eb1-92e7-ca2bd1292272-maas-api-tls\") pod \"maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf\" (UID: \"5085ae9b-40a4-4eb1-92e7-ca2bd1292272\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf"
Jul 30 01:30:30.290315 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.290284    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5085ae9b-40a4-4eb1-92e7-ca2bd1292272-maas-api-tls\") pod \"maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf\" (UID: \"5085ae9b-40a4-4eb1-92e7-ca2bd1292272\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf"
Jul 30 01:30:30.380798 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.380762    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4"]
Jul 30 01:30:30.384454 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.384438    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4"
Jul 30 01:30:30.388992 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.388961    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-b11aa3\""
Jul 30 01:30:30.389139 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.389118    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-b11aa3-dockercfg-vxnpl\""
Jul 30 01:30:30.398219 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.398192    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4"]
Jul 30 01:30:30.411578 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.411554    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5"]
Jul 30 01:30:30.415429 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.415409    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5"
Jul 30 01:30:30.425076 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.425051    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5"]
Jul 30 01:30:30.468116 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.468086    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf"
Jul 30 01:30:30.490148 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.490113    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07ca31c0-9c48-418b-b2ff-adf9d1beed3e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5\" (UID: \"07ca31c0-9c48-418b-b2ff-adf9d1beed3e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5"
Jul 30 01:30:30.490290 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.490209    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05df3a11-e604-4db0-bcc7-16d7ccca7ca1-plugins-config-volume\") pod \"payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4\" (UID: \"05df3a11-e604-4db0-bcc7-16d7ccca7ca1\") " pod="openshift-ingress/payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4"
Jul 30 01:30:30.490733 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.490690    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k64g\" (UniqueName: \"kubernetes.io/projected/05df3a11-e604-4db0-bcc7-16d7ccca7ca1-kube-api-access-4k64g\") pod \"payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4\" (UID: \"05df3a11-e604-4db0-bcc7-16d7ccca7ca1\") " pod="openshift-ingress/payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4"
Jul 30 01:30:30.491243 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.491216    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p798t\" (UniqueName: \"kubernetes.io/projected/07ca31c0-9c48-418b-b2ff-adf9d1beed3e-kube-api-access-p798t\") pod \"payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5\" (UID: \"07ca31c0-9c48-418b-b2ff-adf9d1beed3e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5"
Jul 30 01:30:30.594335 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.592191    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p798t\" (UniqueName: \"kubernetes.io/projected/07ca31c0-9c48-418b-b2ff-adf9d1beed3e-kube-api-access-p798t\") pod \"payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5\" (UID: \"07ca31c0-9c48-418b-b2ff-adf9d1beed3e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5"
Jul 30 01:30:30.594335 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.592273    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07ca31c0-9c48-418b-b2ff-adf9d1beed3e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5\" (UID: \"07ca31c0-9c48-418b-b2ff-adf9d1beed3e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5"
Jul 30 01:30:30.594335 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.592310    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05df3a11-e604-4db0-bcc7-16d7ccca7ca1-plugins-config-volume\") pod \"payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4\" (UID: \"05df3a11-e604-4db0-bcc7-16d7ccca7ca1\") " pod="openshift-ingress/payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4"
Jul 30 01:30:30.594335 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.592365    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k64g\" (UniqueName: \"kubernetes.io/projected/05df3a11-e604-4db0-bcc7-16d7ccca7ca1-kube-api-access-4k64g\") pod \"payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4\" (UID: \"05df3a11-e604-4db0-bcc7-16d7ccca7ca1\") " pod="openshift-ingress/payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4"
Jul 30 01:30:30.594335 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.593958    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07ca31c0-9c48-418b-b2ff-adf9d1beed3e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5\" (UID: \"07ca31c0-9c48-418b-b2ff-adf9d1beed3e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5"
Jul 30 01:30:30.594993 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.594749    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05df3a11-e604-4db0-bcc7-16d7ccca7ca1-plugins-config-volume\") pod \"payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4\" (UID: \"05df3a11-e604-4db0-bcc7-16d7ccca7ca1\") " pod="openshift-ingress/payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4"
Jul 30 01:30:30.601103 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.601079    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p798t\" (UniqueName: \"kubernetes.io/projected/07ca31c0-9c48-418b-b2ff-adf9d1beed3e-kube-api-access-p798t\") pod \"payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5\" (UID: \"07ca31c0-9c48-418b-b2ff-adf9d1beed3e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5"
Jul 30 01:30:30.601827 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.601806    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k64g\" (UniqueName: \"kubernetes.io/projected/05df3a11-e604-4db0-bcc7-16d7ccca7ca1-kube-api-access-4k64g\") pod \"payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4\" (UID: \"05df3a11-e604-4db0-bcc7-16d7ccca7ca1\") " pod="openshift-ingress/payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4"
Jul 30 01:30:30.624227 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.624202    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf"]
Jul 30 01:30:30.626173 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:30:30.626144    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5085ae9b_40a4_4eb1_92e7_ca2bd1292272.slice/crio-8ec8c905d44b45f6ed166ec1cbd454eab32649b165110a43df1f368931f88723 WatchSource:0}: Error finding container 8ec8c905d44b45f6ed166ec1cbd454eab32649b165110a43df1f368931f88723: Status 404 returned error can't find the container with id 8ec8c905d44b45f6ed166ec1cbd454eab32649b165110a43df1f368931f88723
Jul 30 01:30:30.699437 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.699405    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4"
Jul 30 01:30:30.728142 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.728118    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5"
Jul 30 01:30:30.858549 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:30.858172    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4"]
Jul 30 01:30:30.859493 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:30:30.859460    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05df3a11_e604_4db0_bcc7_16d7ccca7ca1.slice/crio-9d4a01add211535fc87483a7e511897f1f1d9b1d8e5b13e7e8c9c4543df17f20 WatchSource:0}: Error finding container 9d4a01add211535fc87483a7e511897f1f1d9b1d8e5b13e7e8c9c4543df17f20: Status 404 returned error can't find the container with id 9d4a01add211535fc87483a7e511897f1f1d9b1d8e5b13e7e8c9c4543df17f20
Jul 30 01:30:31.097400 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:31.097270    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5"]
Jul 30 01:30:31.099063 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:30:31.099032    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07ca31c0_9c48_418b_b2ff_adf9d1beed3e.slice/crio-462a0f9aacc0253b8c148cb2dc90f3bd2ed275e439c8b1f8bcb96d8f28c04cb6 WatchSource:0}: Error finding container 462a0f9aacc0253b8c148cb2dc90f3bd2ed275e439c8b1f8bcb96d8f28c04cb6: Status 404 returned error can't find the container with id 462a0f9aacc0253b8c148cb2dc90f3bd2ed275e439c8b1f8bcb96d8f28c04cb6
Jul 30 01:30:31.499045 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:31.499010    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5" event={"ID":"07ca31c0-9c48-418b-b2ff-adf9d1beed3e","Type":"ContainerStarted","Data":"a7152b9b354aea214059b24e3f3a8afefb1dc9d9e1f5d12218a54a668a381083"}
Jul 30 01:30:31.499299 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:31.499278    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5" event={"ID":"07ca31c0-9c48-418b-b2ff-adf9d1beed3e","Type":"ContainerStarted","Data":"462a0f9aacc0253b8c148cb2dc90f3bd2ed275e439c8b1f8bcb96d8f28c04cb6"}
Jul 30 01:30:31.499443 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:31.499421    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5"
Jul 30 01:30:31.500936 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:31.500913    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4" event={"ID":"05df3a11-e604-4db0-bcc7-16d7ccca7ca1","Type":"ContainerStarted","Data":"f2da4dcbc4784fba7e9dfd0ae6882579219b57e4089217c1d50937030d952320"}
Jul 30 01:30:31.501054 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:31.500959    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4" event={"ID":"05df3a11-e604-4db0-bcc7-16d7ccca7ca1","Type":"ContainerStarted","Data":"9d4a01add211535fc87483a7e511897f1f1d9b1d8e5b13e7e8c9c4543df17f20"}
Jul 30 01:30:31.501054 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:31.500988    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4"
Jul 30 01:30:31.502752 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:31.502729    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf" event={"ID":"5085ae9b-40a4-4eb1-92e7-ca2bd1292272","Type":"ContainerStarted","Data":"aca6a9d8cb9dbec60b91b66d1f839207e28e0275178b1e8f1dee619820974a73"}
Jul 30 01:30:31.502852 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:31.502773    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf" event={"ID":"5085ae9b-40a4-4eb1-92e7-ca2bd1292272","Type":"ContainerStarted","Data":"8ec8c905d44b45f6ed166ec1cbd454eab32649b165110a43df1f368931f88723"}
Jul 30 01:30:31.502852 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:31.502826    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf"
Jul 30 01:30:31.519129 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:31.519072    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5" podStartSLOduration=1.5190586879999999 podStartE2EDuration="1.519058688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:30:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:30:31.517004399 +0000 UTC m=+2818.739738218" watchObservedRunningTime="2026-07-30 01:30:31.519058688 +0000 UTC m=+2818.741792496"
Jul 30 01:30:31.535249 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:31.535195    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4" podStartSLOduration=1.5351772559999999 podStartE2EDuration="1.535177256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:30:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:30:31.532645272 +0000 UTC m=+2818.755379090" watchObservedRunningTime="2026-07-30 01:30:31.535177256 +0000 UTC m=+2818.757911075"
Jul 30 01:30:31.551092 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:31.551038    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf" podStartSLOduration=2.19364546 podStartE2EDuration="2.551018371s" totalImagesPullingTime="357.372911ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:30:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:30:31.549231162 +0000 UTC m=+2818.771965000" watchObservedRunningTime="2026-07-30 01:30:31.551018371 +0000 UTC m=+2818.773752193"
Jul 30 01:30:33.641507 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:33.641474    2546 scope.go:117] "RemoveContainer" containerID="1539eeb33996ce8cde87fcb7d52557a19c291b2737af9498bc77fce6cb05ed9d"
Jul 30 01:30:37.514820 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:37.514787    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf"
Jul 30 01:30:42.510401 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:42.510370    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4"
Jul 30 01:30:43.515857 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:30:43.515826    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5"
Jul 30 01:31:11.971175 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:11.971140    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx"]
Jul 30 01:31:11.972155 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:11.972125    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx" podUID="7651aa76-d4bb-4f06-96ae-afe66836112c" containerName="maas-api" containerID="cri-o://f5c8a888362b1353ba773e8571fde70a65ca2ab9b7c8973082b5c8c010ebf5cb" gracePeriod=30
Jul 30 01:31:12.023008 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.022968    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t"]
Jul 30 01:31:12.023277 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.023225    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t" podUID="15828e66-2547-4c5e-b94f-ff2ee1b75d86" containerName="payload-processing" containerID="cri-o://76e1c71948c3eb275b8c8742d1d911c0880963c76f9ab416925ef1b4edddce4e" gracePeriod=30
Jul 30 01:31:12.030829 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.030795    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p"]
Jul 30 01:31:12.031051 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.031022    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p" podUID="a3046f61-9c6c-41eb-87e3-3046708f4c7d" containerName="payload-pre-processing" containerID="cri-o://2187ad56dcf512cf4a443edbd0b11db45a3bfa6034a4810a8729d8baa9daceda" gracePeriod=30
Jul 30 01:31:12.310941 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.310913    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t"
Jul 30 01:31:12.345210 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.345186    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx"
Jul 30 01:31:12.349382 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.349354    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p"
Jul 30 01:31:12.369116 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.369083    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7651aa76-d4bb-4f06-96ae-afe66836112c-maas-api-tls\") pod \"7651aa76-d4bb-4f06-96ae-afe66836112c\" (UID: \"7651aa76-d4bb-4f06-96ae-afe66836112c\") "
Jul 30 01:31:12.369116 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.369118    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fcfj\" (UniqueName: \"kubernetes.io/projected/7651aa76-d4bb-4f06-96ae-afe66836112c-kube-api-access-6fcfj\") pod \"7651aa76-d4bb-4f06-96ae-afe66836112c\" (UID: \"7651aa76-d4bb-4f06-96ae-afe66836112c\") "
Jul 30 01:31:12.369676 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.369155    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t94rv\" (UniqueName: \"kubernetes.io/projected/15828e66-2547-4c5e-b94f-ff2ee1b75d86-kube-api-access-t94rv\") pod \"15828e66-2547-4c5e-b94f-ff2ee1b75d86\" (UID: \"15828e66-2547-4c5e-b94f-ff2ee1b75d86\") "
Jul 30 01:31:12.369676 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.369220    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7651aa76-d4bb-4f06-96ae-afe66836112c-openshift-service-ca\") pod \"7651aa76-d4bb-4f06-96ae-afe66836112c\" (UID: \"7651aa76-d4bb-4f06-96ae-afe66836112c\") "
Jul 30 01:31:12.369676 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.369280    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15828e66-2547-4c5e-b94f-ff2ee1b75d86-plugins-config-volume\") pod \"15828e66-2547-4c5e-b94f-ff2ee1b75d86\" (UID: \"15828e66-2547-4c5e-b94f-ff2ee1b75d86\") "
Jul 30 01:31:12.369676 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.369313    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3046f61-9c6c-41eb-87e3-3046708f4c7d-plugins-config-volume\") pod \"a3046f61-9c6c-41eb-87e3-3046708f4c7d\" (UID: \"a3046f61-9c6c-41eb-87e3-3046708f4c7d\") "
Jul 30 01:31:12.369676 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.369372    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fw5fs\" (UniqueName: \"kubernetes.io/projected/a3046f61-9c6c-41eb-87e3-3046708f4c7d-kube-api-access-fw5fs\") pod \"a3046f61-9c6c-41eb-87e3-3046708f4c7d\" (UID: \"a3046f61-9c6c-41eb-87e3-3046708f4c7d\") "
Jul 30 01:31:12.370552 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.370422    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15828e66-2547-4c5e-b94f-ff2ee1b75d86-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "15828e66-2547-4c5e-b94f-ff2ee1b75d86" (UID: "15828e66-2547-4c5e-b94f-ff2ee1b75d86"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:31:12.370552 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.370441    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3046f61-9c6c-41eb-87e3-3046708f4c7d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a3046f61-9c6c-41eb-87e3-3046708f4c7d" (UID: "a3046f61-9c6c-41eb-87e3-3046708f4c7d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:31:12.370923 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.370897    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7651aa76-d4bb-4f06-96ae-afe66836112c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7651aa76-d4bb-4f06-96ae-afe66836112c" (UID: "7651aa76-d4bb-4f06-96ae-afe66836112c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:31:12.371887 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.371853    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7651aa76-d4bb-4f06-96ae-afe66836112c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7651aa76-d4bb-4f06-96ae-afe66836112c" (UID: "7651aa76-d4bb-4f06-96ae-afe66836112c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:31:12.371992 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.371919    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7651aa76-d4bb-4f06-96ae-afe66836112c-kube-api-access-6fcfj" (OuterVolumeSpecName: "kube-api-access-6fcfj") pod "7651aa76-d4bb-4f06-96ae-afe66836112c" (UID: "7651aa76-d4bb-4f06-96ae-afe66836112c"). InnerVolumeSpecName "kube-api-access-6fcfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:31:12.372144 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.372120    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3046f61-9c6c-41eb-87e3-3046708f4c7d-kube-api-access-fw5fs" (OuterVolumeSpecName: "kube-api-access-fw5fs") pod "a3046f61-9c6c-41eb-87e3-3046708f4c7d" (UID: "a3046f61-9c6c-41eb-87e3-3046708f4c7d"). InnerVolumeSpecName "kube-api-access-fw5fs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:31:12.372266 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.372247    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15828e66-2547-4c5e-b94f-ff2ee1b75d86-kube-api-access-t94rv" (OuterVolumeSpecName: "kube-api-access-t94rv") pod "15828e66-2547-4c5e-b94f-ff2ee1b75d86" (UID: "15828e66-2547-4c5e-b94f-ff2ee1b75d86"). InnerVolumeSpecName "kube-api-access-t94rv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:31:12.470650 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.470617    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7651aa76-d4bb-4f06-96ae-afe66836112c-openshift-service-ca\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:31:12.470650 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.470643    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15828e66-2547-4c5e-b94f-ff2ee1b75d86-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:31:12.470650 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.470653    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3046f61-9c6c-41eb-87e3-3046708f4c7d-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:31:12.470899 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.470662    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fw5fs\" (UniqueName: \"kubernetes.io/projected/a3046f61-9c6c-41eb-87e3-3046708f4c7d-kube-api-access-fw5fs\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:31:12.470899 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.470673    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7651aa76-d4bb-4f06-96ae-afe66836112c-maas-api-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:31:12.470899 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.470681    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fcfj\" (UniqueName: \"kubernetes.io/projected/7651aa76-d4bb-4f06-96ae-afe66836112c-kube-api-access-6fcfj\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:31:12.470899 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.470697    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t94rv\" (UniqueName: \"kubernetes.io/projected/15828e66-2547-4c5e-b94f-ff2ee1b75d86-kube-api-access-t94rv\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:31:12.719544 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.719494    2546 generic.go:356] "Generic (PLEG): container finished" podID="7651aa76-d4bb-4f06-96ae-afe66836112c" containerID="f5c8a888362b1353ba773e8571fde70a65ca2ab9b7c8973082b5c8c010ebf5cb" exitCode=0
Jul 30 01:31:12.719739 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.719578    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx"
Jul 30 01:31:12.719739 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.719579    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx" event={"ID":"7651aa76-d4bb-4f06-96ae-afe66836112c","Type":"ContainerDied","Data":"f5c8a888362b1353ba773e8571fde70a65ca2ab9b7c8973082b5c8c010ebf5cb"}
Jul 30 01:31:12.719739 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.719732    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx" event={"ID":"7651aa76-d4bb-4f06-96ae-afe66836112c","Type":"ContainerDied","Data":"3fa745280b89352c3bb17e471216819269af61e607e15622b0ec03c20d4ba645"}
Jul 30 01:31:12.719918 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.719766    2546 scope.go:117] "RemoveContainer" containerID="f5c8a888362b1353ba773e8571fde70a65ca2ab9b7c8973082b5c8c010ebf5cb"
Jul 30 01:31:12.721202 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.721175    2546 generic.go:356] "Generic (PLEG): container finished" podID="15828e66-2547-4c5e-b94f-ff2ee1b75d86" containerID="76e1c71948c3eb275b8c8742d1d911c0880963c76f9ab416925ef1b4edddce4e" exitCode=0
Jul 30 01:31:12.721323 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.721236    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t"
Jul 30 01:31:12.721323 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.721258    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t" event={"ID":"15828e66-2547-4c5e-b94f-ff2ee1b75d86","Type":"ContainerDied","Data":"76e1c71948c3eb275b8c8742d1d911c0880963c76f9ab416925ef1b4edddce4e"}
Jul 30 01:31:12.721323 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.721303    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t" event={"ID":"15828e66-2547-4c5e-b94f-ff2ee1b75d86","Type":"ContainerDied","Data":"0a9a61b529ac6b9575529375b380873608d37a00e91a228b784723407e8c3de6"}
Jul 30 01:31:12.722638 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.722618    2546 generic.go:356] "Generic (PLEG): container finished" podID="a3046f61-9c6c-41eb-87e3-3046708f4c7d" containerID="2187ad56dcf512cf4a443edbd0b11db45a3bfa6034a4810a8729d8baa9daceda" exitCode=0
Jul 30 01:31:12.722766 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.722644    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p" event={"ID":"a3046f61-9c6c-41eb-87e3-3046708f4c7d","Type":"ContainerDied","Data":"2187ad56dcf512cf4a443edbd0b11db45a3bfa6034a4810a8729d8baa9daceda"}
Jul 30 01:31:12.722766 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.722668    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p"
Jul 30 01:31:12.722766 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.722671    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p" event={"ID":"a3046f61-9c6c-41eb-87e3-3046708f4c7d","Type":"ContainerDied","Data":"d698b03a9f34116b271fac29470ce0a730be5fadbdd8a7bdca8ab3e7c76c78e4"}
Jul 30 01:31:12.731946 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.731929    2546 scope.go:117] "RemoveContainer" containerID="f5c8a888362b1353ba773e8571fde70a65ca2ab9b7c8973082b5c8c010ebf5cb"
Jul 30 01:31:12.732227 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:31:12.732204    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5c8a888362b1353ba773e8571fde70a65ca2ab9b7c8973082b5c8c010ebf5cb\": container with ID starting with f5c8a888362b1353ba773e8571fde70a65ca2ab9b7c8973082b5c8c010ebf5cb not found: ID does not exist" containerID="f5c8a888362b1353ba773e8571fde70a65ca2ab9b7c8973082b5c8c010ebf5cb"
Jul 30 01:31:12.732324 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.732233    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5c8a888362b1353ba773e8571fde70a65ca2ab9b7c8973082b5c8c010ebf5cb"} err="failed to get container status \"f5c8a888362b1353ba773e8571fde70a65ca2ab9b7c8973082b5c8c010ebf5cb\": rpc error: code = NotFound desc = could not find container \"f5c8a888362b1353ba773e8571fde70a65ca2ab9b7c8973082b5c8c010ebf5cb\": container with ID starting with f5c8a888362b1353ba773e8571fde70a65ca2ab9b7c8973082b5c8c010ebf5cb not found: ID does not exist"
Jul 30 01:31:12.732324 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.732248    2546 scope.go:117] "RemoveContainer" containerID="76e1c71948c3eb275b8c8742d1d911c0880963c76f9ab416925ef1b4edddce4e"
Jul 30 01:31:12.740678 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.740646    2546 scope.go:117] "RemoveContainer" containerID="76e1c71948c3eb275b8c8742d1d911c0880963c76f9ab416925ef1b4edddce4e"
Jul 30 01:31:12.740930 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:31:12.740894    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76e1c71948c3eb275b8c8742d1d911c0880963c76f9ab416925ef1b4edddce4e\": container with ID starting with 76e1c71948c3eb275b8c8742d1d911c0880963c76f9ab416925ef1b4edddce4e not found: ID does not exist" containerID="76e1c71948c3eb275b8c8742d1d911c0880963c76f9ab416925ef1b4edddce4e"
Jul 30 01:31:12.740930 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.740919    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76e1c71948c3eb275b8c8742d1d911c0880963c76f9ab416925ef1b4edddce4e"} err="failed to get container status \"76e1c71948c3eb275b8c8742d1d911c0880963c76f9ab416925ef1b4edddce4e\": rpc error: code = NotFound desc = could not find container \"76e1c71948c3eb275b8c8742d1d911c0880963c76f9ab416925ef1b4edddce4e\": container with ID starting with 76e1c71948c3eb275b8c8742d1d911c0880963c76f9ab416925ef1b4edddce4e not found: ID does not exist"
Jul 30 01:31:12.741055 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.740936    2546 scope.go:117] "RemoveContainer" containerID="2187ad56dcf512cf4a443edbd0b11db45a3bfa6034a4810a8729d8baa9daceda"
Jul 30 01:31:12.744660 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.744638    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx"]
Jul 30 01:31:12.750614 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.750327    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5ca190-7657548f9f-84dsx"]
Jul 30 01:31:12.752169 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.752154    2546 scope.go:117] "RemoveContainer" containerID="2187ad56dcf512cf4a443edbd0b11db45a3bfa6034a4810a8729d8baa9daceda"
Jul 30 01:31:12.752463 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:31:12.752443    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2187ad56dcf512cf4a443edbd0b11db45a3bfa6034a4810a8729d8baa9daceda\": container with ID starting with 2187ad56dcf512cf4a443edbd0b11db45a3bfa6034a4810a8729d8baa9daceda not found: ID does not exist" containerID="2187ad56dcf512cf4a443edbd0b11db45a3bfa6034a4810a8729d8baa9daceda"
Jul 30 01:31:12.752543 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.752469    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2187ad56dcf512cf4a443edbd0b11db45a3bfa6034a4810a8729d8baa9daceda"} err="failed to get container status \"2187ad56dcf512cf4a443edbd0b11db45a3bfa6034a4810a8729d8baa9daceda\": rpc error: code = NotFound desc = could not find container \"2187ad56dcf512cf4a443edbd0b11db45a3bfa6034a4810a8729d8baa9daceda\": container with ID starting with 2187ad56dcf512cf4a443edbd0b11db45a3bfa6034a4810a8729d8baa9daceda not found: ID does not exist"
Jul 30 01:31:12.760891 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.760869    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p"]
Jul 30 01:31:12.764930 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.764910    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5ca190-846b74dbfb-tqj9p"]
Jul 30 01:31:12.777774 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.777752    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t"]
Jul 30 01:31:12.783869 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:12.783848    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5ca190-d58774db6-fgk9t"]
Jul 30 01:31:13.358501 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:13.358423    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15828e66-2547-4c5e-b94f-ff2ee1b75d86" path="/var/lib/kubelet/pods/15828e66-2547-4c5e-b94f-ff2ee1b75d86/volumes"
Jul 30 01:31:13.358883 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:13.358839    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7651aa76-d4bb-4f06-96ae-afe66836112c" path="/var/lib/kubelet/pods/7651aa76-d4bb-4f06-96ae-afe66836112c/volumes"
Jul 30 01:31:13.359150 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:13.359137    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3046f61-9c6c-41eb-87e3-3046708f4c7d" path="/var/lib/kubelet/pods/a3046f61-9c6c-41eb-87e3-3046708f4c7d/volumes"
Jul 30 01:31:34.405018 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.404980    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf"]
Jul 30 01:31:34.405505 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.405247    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf" podUID="5085ae9b-40a4-4eb1-92e7-ca2bd1292272" containerName="maas-api" containerID="cri-o://aca6a9d8cb9dbec60b91b66d1f839207e28e0275178b1e8f1dee619820974a73" gracePeriod=30
Jul 30 01:31:34.459615 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.459584    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4"]
Jul 30 01:31:34.459815 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.459790    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4" podUID="05df3a11-e604-4db0-bcc7-16d7ccca7ca1" containerName="payload-processing" containerID="cri-o://f2da4dcbc4784fba7e9dfd0ae6882579219b57e4089217c1d50937030d952320" gracePeriod=30
Jul 30 01:31:34.468258 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.468227    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5"]
Jul 30 01:31:34.468500 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.468468    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5" podUID="07ca31c0-9c48-418b-b2ff-adf9d1beed3e" containerName="payload-pre-processing" containerID="cri-o://a7152b9b354aea214059b24e3f3a8afefb1dc9d9e1f5d12218a54a668a381083" gracePeriod=30
Jul 30 01:31:34.744649 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.744621    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf"
Jul 30 01:31:34.768410 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.768378    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4"
Jul 30 01:31:34.777262 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.777239    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5"
Jul 30 01:31:34.840818 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.840768    2546 generic.go:356] "Generic (PLEG): container finished" podID="5085ae9b-40a4-4eb1-92e7-ca2bd1292272" containerID="aca6a9d8cb9dbec60b91b66d1f839207e28e0275178b1e8f1dee619820974a73" exitCode=0
Jul 30 01:31:34.840818 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.840817    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf"
Jul 30 01:31:34.841039 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.840833    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf" event={"ID":"5085ae9b-40a4-4eb1-92e7-ca2bd1292272","Type":"ContainerDied","Data":"aca6a9d8cb9dbec60b91b66d1f839207e28e0275178b1e8f1dee619820974a73"}
Jul 30 01:31:34.841039 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.840893    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf" event={"ID":"5085ae9b-40a4-4eb1-92e7-ca2bd1292272","Type":"ContainerDied","Data":"8ec8c905d44b45f6ed166ec1cbd454eab32649b165110a43df1f368931f88723"}
Jul 30 01:31:34.841039 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.840918    2546 scope.go:117] "RemoveContainer" containerID="aca6a9d8cb9dbec60b91b66d1f839207e28e0275178b1e8f1dee619820974a73"
Jul 30 01:31:34.842198 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.842177    2546 generic.go:356] "Generic (PLEG): container finished" podID="07ca31c0-9c48-418b-b2ff-adf9d1beed3e" containerID="a7152b9b354aea214059b24e3f3a8afefb1dc9d9e1f5d12218a54a668a381083" exitCode=0
Jul 30 01:31:34.842288 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.842208    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5" event={"ID":"07ca31c0-9c48-418b-b2ff-adf9d1beed3e","Type":"ContainerDied","Data":"a7152b9b354aea214059b24e3f3a8afefb1dc9d9e1f5d12218a54a668a381083"}
Jul 30 01:31:34.842288 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.842261    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5" event={"ID":"07ca31c0-9c48-418b-b2ff-adf9d1beed3e","Type":"ContainerDied","Data":"462a0f9aacc0253b8c148cb2dc90f3bd2ed275e439c8b1f8bcb96d8f28c04cb6"}
Jul 30 01:31:34.842405 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.842219    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5"
Jul 30 01:31:34.843662 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.843639    2546 generic.go:356] "Generic (PLEG): container finished" podID="05df3a11-e604-4db0-bcc7-16d7ccca7ca1" containerID="f2da4dcbc4784fba7e9dfd0ae6882579219b57e4089217c1d50937030d952320" exitCode=0
Jul 30 01:31:34.843779 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.843672    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4" event={"ID":"05df3a11-e604-4db0-bcc7-16d7ccca7ca1","Type":"ContainerDied","Data":"f2da4dcbc4784fba7e9dfd0ae6882579219b57e4089217c1d50937030d952320"}
Jul 30 01:31:34.843779 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.843680    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4"
Jul 30 01:31:34.843779 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.843707    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4" event={"ID":"05df3a11-e604-4db0-bcc7-16d7ccca7ca1","Type":"ContainerDied","Data":"9d4a01add211535fc87483a7e511897f1f1d9b1d8e5b13e7e8c9c4543df17f20"}
Jul 30 01:31:34.850388 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.850367    2546 scope.go:117] "RemoveContainer" containerID="aca6a9d8cb9dbec60b91b66d1f839207e28e0275178b1e8f1dee619820974a73"
Jul 30 01:31:34.850694 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:31:34.850670    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aca6a9d8cb9dbec60b91b66d1f839207e28e0275178b1e8f1dee619820974a73\": container with ID starting with aca6a9d8cb9dbec60b91b66d1f839207e28e0275178b1e8f1dee619820974a73 not found: ID does not exist" containerID="aca6a9d8cb9dbec60b91b66d1f839207e28e0275178b1e8f1dee619820974a73"
Jul 30 01:31:34.850764 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.850702    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aca6a9d8cb9dbec60b91b66d1f839207e28e0275178b1e8f1dee619820974a73"} err="failed to get container status \"aca6a9d8cb9dbec60b91b66d1f839207e28e0275178b1e8f1dee619820974a73\": rpc error: code = NotFound desc = could not find container \"aca6a9d8cb9dbec60b91b66d1f839207e28e0275178b1e8f1dee619820974a73\": container with ID starting with aca6a9d8cb9dbec60b91b66d1f839207e28e0275178b1e8f1dee619820974a73 not found: ID does not exist"
Jul 30 01:31:34.850764 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.850718    2546 scope.go:117] "RemoveContainer" containerID="a7152b9b354aea214059b24e3f3a8afefb1dc9d9e1f5d12218a54a668a381083"
Jul 30 01:31:34.859033 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.859010    2546 scope.go:117] "RemoveContainer" containerID="a7152b9b354aea214059b24e3f3a8afefb1dc9d9e1f5d12218a54a668a381083"
Jul 30 01:31:34.859300 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:31:34.859282    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7152b9b354aea214059b24e3f3a8afefb1dc9d9e1f5d12218a54a668a381083\": container with ID starting with a7152b9b354aea214059b24e3f3a8afefb1dc9d9e1f5d12218a54a668a381083 not found: ID does not exist" containerID="a7152b9b354aea214059b24e3f3a8afefb1dc9d9e1f5d12218a54a668a381083"
Jul 30 01:31:34.859354 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.859307    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7152b9b354aea214059b24e3f3a8afefb1dc9d9e1f5d12218a54a668a381083"} err="failed to get container status \"a7152b9b354aea214059b24e3f3a8afefb1dc9d9e1f5d12218a54a668a381083\": rpc error: code = NotFound desc = could not find container \"a7152b9b354aea214059b24e3f3a8afefb1dc9d9e1f5d12218a54a668a381083\": container with ID starting with a7152b9b354aea214059b24e3f3a8afefb1dc9d9e1f5d12218a54a668a381083 not found: ID does not exist"
Jul 30 01:31:34.859354 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.859323    2546 scope.go:117] "RemoveContainer" containerID="f2da4dcbc4784fba7e9dfd0ae6882579219b57e4089217c1d50937030d952320"
Jul 30 01:31:34.867740 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.867723    2546 scope.go:117] "RemoveContainer" containerID="f2da4dcbc4784fba7e9dfd0ae6882579219b57e4089217c1d50937030d952320"
Jul 30 01:31:34.867996 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:31:34.867971    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2da4dcbc4784fba7e9dfd0ae6882579219b57e4089217c1d50937030d952320\": container with ID starting with f2da4dcbc4784fba7e9dfd0ae6882579219b57e4089217c1d50937030d952320 not found: ID does not exist" containerID="f2da4dcbc4784fba7e9dfd0ae6882579219b57e4089217c1d50937030d952320"
Jul 30 01:31:34.868055 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.867999    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2da4dcbc4784fba7e9dfd0ae6882579219b57e4089217c1d50937030d952320"} err="failed to get container status \"f2da4dcbc4784fba7e9dfd0ae6882579219b57e4089217c1d50937030d952320\": rpc error: code = NotFound desc = could not find container \"f2da4dcbc4784fba7e9dfd0ae6882579219b57e4089217c1d50937030d952320\": container with ID starting with f2da4dcbc4784fba7e9dfd0ae6882579219b57e4089217c1d50937030d952320 not found: ID does not exist"
Jul 30 01:31:34.882387 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.882360    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxmrv\" (UniqueName: \"kubernetes.io/projected/5085ae9b-40a4-4eb1-92e7-ca2bd1292272-kube-api-access-qxmrv\") pod \"5085ae9b-40a4-4eb1-92e7-ca2bd1292272\" (UID: \"5085ae9b-40a4-4eb1-92e7-ca2bd1292272\") "
Jul 30 01:31:34.882500 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.882411    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5085ae9b-40a4-4eb1-92e7-ca2bd1292272-maas-api-tls\") pod \"5085ae9b-40a4-4eb1-92e7-ca2bd1292272\" (UID: \"5085ae9b-40a4-4eb1-92e7-ca2bd1292272\") "
Jul 30 01:31:34.882500 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.882439    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4k64g\" (UniqueName: \"kubernetes.io/projected/05df3a11-e604-4db0-bcc7-16d7ccca7ca1-kube-api-access-4k64g\") pod \"05df3a11-e604-4db0-bcc7-16d7ccca7ca1\" (UID: \"05df3a11-e604-4db0-bcc7-16d7ccca7ca1\") "
Jul 30 01:31:34.882636 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.882504    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p798t\" (UniqueName: \"kubernetes.io/projected/07ca31c0-9c48-418b-b2ff-adf9d1beed3e-kube-api-access-p798t\") pod \"07ca31c0-9c48-418b-b2ff-adf9d1beed3e\" (UID: \"07ca31c0-9c48-418b-b2ff-adf9d1beed3e\") "
Jul 30 01:31:34.882636 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.882552    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5085ae9b-40a4-4eb1-92e7-ca2bd1292272-openshift-service-ca\") pod \"5085ae9b-40a4-4eb1-92e7-ca2bd1292272\" (UID: \"5085ae9b-40a4-4eb1-92e7-ca2bd1292272\") "
Jul 30 01:31:34.882636 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.882578    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07ca31c0-9c48-418b-b2ff-adf9d1beed3e-plugins-config-volume\") pod \"07ca31c0-9c48-418b-b2ff-adf9d1beed3e\" (UID: \"07ca31c0-9c48-418b-b2ff-adf9d1beed3e\") "
Jul 30 01:31:34.882636 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.882602    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05df3a11-e604-4db0-bcc7-16d7ccca7ca1-plugins-config-volume\") pod \"05df3a11-e604-4db0-bcc7-16d7ccca7ca1\" (UID: \"05df3a11-e604-4db0-bcc7-16d7ccca7ca1\") "
Jul 30 01:31:34.882975 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.882947    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5085ae9b-40a4-4eb1-92e7-ca2bd1292272-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5085ae9b-40a4-4eb1-92e7-ca2bd1292272" (UID: "5085ae9b-40a4-4eb1-92e7-ca2bd1292272"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:31:34.883056 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.882968    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07ca31c0-9c48-418b-b2ff-adf9d1beed3e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "07ca31c0-9c48-418b-b2ff-adf9d1beed3e" (UID: "07ca31c0-9c48-418b-b2ff-adf9d1beed3e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:31:34.883056 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.883037    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05df3a11-e604-4db0-bcc7-16d7ccca7ca1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "05df3a11-e604-4db0-bcc7-16d7ccca7ca1" (UID: "05df3a11-e604-4db0-bcc7-16d7ccca7ca1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:31:34.884641 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.884513    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05df3a11-e604-4db0-bcc7-16d7ccca7ca1-kube-api-access-4k64g" (OuterVolumeSpecName: "kube-api-access-4k64g") pod "05df3a11-e604-4db0-bcc7-16d7ccca7ca1" (UID: "05df3a11-e604-4db0-bcc7-16d7ccca7ca1"). InnerVolumeSpecName "kube-api-access-4k64g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:31:34.884641 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.884618    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5085ae9b-40a4-4eb1-92e7-ca2bd1292272-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5085ae9b-40a4-4eb1-92e7-ca2bd1292272" (UID: "5085ae9b-40a4-4eb1-92e7-ca2bd1292272"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:31:34.884761 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.884616    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07ca31c0-9c48-418b-b2ff-adf9d1beed3e-kube-api-access-p798t" (OuterVolumeSpecName: "kube-api-access-p798t") pod "07ca31c0-9c48-418b-b2ff-adf9d1beed3e" (UID: "07ca31c0-9c48-418b-b2ff-adf9d1beed3e"). InnerVolumeSpecName "kube-api-access-p798t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:31:34.884837 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.884815    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5085ae9b-40a4-4eb1-92e7-ca2bd1292272-kube-api-access-qxmrv" (OuterVolumeSpecName: "kube-api-access-qxmrv") pod "5085ae9b-40a4-4eb1-92e7-ca2bd1292272" (UID: "5085ae9b-40a4-4eb1-92e7-ca2bd1292272"). InnerVolumeSpecName "kube-api-access-qxmrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:31:34.983694 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.983660    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxmrv\" (UniqueName: \"kubernetes.io/projected/5085ae9b-40a4-4eb1-92e7-ca2bd1292272-kube-api-access-qxmrv\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:31:34.983694 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.983691    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5085ae9b-40a4-4eb1-92e7-ca2bd1292272-maas-api-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:31:34.983909 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.983706    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4k64g\" (UniqueName: \"kubernetes.io/projected/05df3a11-e604-4db0-bcc7-16d7ccca7ca1-kube-api-access-4k64g\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:31:34.983909 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.983718    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p798t\" (UniqueName: \"kubernetes.io/projected/07ca31c0-9c48-418b-b2ff-adf9d1beed3e-kube-api-access-p798t\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:31:34.983909 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.983756    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5085ae9b-40a4-4eb1-92e7-ca2bd1292272-openshift-service-ca\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:31:34.983909 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.983770    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07ca31c0-9c48-418b-b2ff-adf9d1beed3e-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:31:34.983909 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:34.983783    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05df3a11-e604-4db0-bcc7-16d7ccca7ca1-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:31:35.167981 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:35.167943    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf"]
Jul 30 01:31:35.172278 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:35.172252    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b11aa3-65c6b4958d-2b8sf"]
Jul 30 01:31:35.186008 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:35.185974    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4"]
Jul 30 01:31:35.194326 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:35.194297    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b11aa3-56f89b656d-26sj4"]
Jul 30 01:31:35.203795 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:35.203770    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5"]
Jul 30 01:31:35.208088 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:35.208064    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b11aa3-84998f9b56-v9zr5"]
Jul 30 01:31:35.360378 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:35.360300    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05df3a11-e604-4db0-bcc7-16d7ccca7ca1" path="/var/lib/kubelet/pods/05df3a11-e604-4db0-bcc7-16d7ccca7ca1/volumes"
Jul 30 01:31:35.360677 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:35.360664    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07ca31c0-9c48-418b-b2ff-adf9d1beed3e" path="/var/lib/kubelet/pods/07ca31c0-9c48-418b-b2ff-adf9d1beed3e/volumes"
Jul 30 01:31:35.361003 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:35.360992    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5085ae9b-40a4-4eb1-92e7-ca2bd1292272" path="/var/lib/kubelet/pods/5085ae9b-40a4-4eb1-92e7-ca2bd1292272/volumes"
Jul 30 01:31:46.563261 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.563230    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc"]
Jul 30 01:31:46.563839 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.563823    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3046f61-9c6c-41eb-87e3-3046708f4c7d" containerName="payload-pre-processing"
Jul 30 01:31:46.563885 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.563842    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3046f61-9c6c-41eb-87e3-3046708f4c7d" containerName="payload-pre-processing"
Jul 30 01:31:46.563885 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.563863    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5085ae9b-40a4-4eb1-92e7-ca2bd1292272" containerName="maas-api"
Jul 30 01:31:46.563885 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.563873    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="5085ae9b-40a4-4eb1-92e7-ca2bd1292272" containerName="maas-api"
Jul 30 01:31:46.563980 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.563891    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15828e66-2547-4c5e-b94f-ff2ee1b75d86" containerName="payload-processing"
Jul 30 01:31:46.563980 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.563900    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="15828e66-2547-4c5e-b94f-ff2ee1b75d86" containerName="payload-processing"
Jul 30 01:31:46.563980 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.563914    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05df3a11-e604-4db0-bcc7-16d7ccca7ca1" containerName="payload-processing"
Jul 30 01:31:46.563980 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.563922    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="05df3a11-e604-4db0-bcc7-16d7ccca7ca1" containerName="payload-processing"
Jul 30 01:31:46.563980 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.563941    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7651aa76-d4bb-4f06-96ae-afe66836112c" containerName="maas-api"
Jul 30 01:31:46.563980 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.563949    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="7651aa76-d4bb-4f06-96ae-afe66836112c" containerName="maas-api"
Jul 30 01:31:46.563980 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.563960    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07ca31c0-9c48-418b-b2ff-adf9d1beed3e" containerName="payload-pre-processing"
Jul 30 01:31:46.563980 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.563968    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="07ca31c0-9c48-418b-b2ff-adf9d1beed3e" containerName="payload-pre-processing"
Jul 30 01:31:46.564218 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.564053    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="15828e66-2547-4c5e-b94f-ff2ee1b75d86" containerName="payload-processing"
Jul 30 01:31:46.564218 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.564068    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="07ca31c0-9c48-418b-b2ff-adf9d1beed3e" containerName="payload-pre-processing"
Jul 30 01:31:46.564218 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.564078    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="05df3a11-e604-4db0-bcc7-16d7ccca7ca1" containerName="payload-processing"
Jul 30 01:31:46.564218 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.564086    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="5085ae9b-40a4-4eb1-92e7-ca2bd1292272" containerName="maas-api"
Jul 30 01:31:46.564218 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.564096    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3046f61-9c6c-41eb-87e3-3046708f4c7d" containerName="payload-pre-processing"
Jul 30 01:31:46.564218 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.564105    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="7651aa76-d4bb-4f06-96ae-afe66836112c" containerName="maas-api"
Jul 30 01:31:46.567473 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.567453    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc"
Jul 30 01:31:46.569867 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.569843    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-55b321-serving-cert\""
Jul 30 01:31:46.577363 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.577331    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc"]
Jul 30 01:31:46.577844 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.577826    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc\" (UID: \"a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc"
Jul 30 01:31:46.577962 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.577939    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2-maas-api-tls\") pod \"maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc\" (UID: \"a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc"
Jul 30 01:31:46.578023 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.578001    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwr6l\" (UniqueName: \"kubernetes.io/projected/a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2-kube-api-access-wwr6l\") pod \"maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc\" (UID: \"a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc"
Jul 30 01:31:46.678507 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.678472    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2-maas-api-tls\") pod \"maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc\" (UID: \"a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc"
Jul 30 01:31:46.678702 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.678561    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwr6l\" (UniqueName: \"kubernetes.io/projected/a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2-kube-api-access-wwr6l\") pod \"maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc\" (UID: \"a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc"
Jul 30 01:31:46.678702 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.678601    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc\" (UID: \"a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc"
Jul 30 01:31:46.679280 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.679261    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc\" (UID: \"a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc"
Jul 30 01:31:46.680995 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.680972    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2-maas-api-tls\") pod \"maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc\" (UID: \"a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc"
Jul 30 01:31:46.688532 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.688498    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwr6l\" (UniqueName: \"kubernetes.io/projected/a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2-kube-api-access-wwr6l\") pod \"maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc\" (UID: \"a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc"
Jul 30 01:31:46.879899 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:46.879792    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc"
Jul 30 01:31:47.054877 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.054832    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc"]
Jul 30 01:31:47.057323 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:31:47.057292    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda44f902c_4e2e_44d2_a3ef_87e8c70cc7c2.slice/crio-9d70eab84c63f2ad20c1845221ac9884f90614286409674a5602dcb893cb056e WatchSource:0}: Error finding container 9d70eab84c63f2ad20c1845221ac9884f90614286409674a5602dcb893cb056e: Status 404 returned error can't find the container with id 9d70eab84c63f2ad20c1845221ac9884f90614286409674a5602dcb893cb056e
Jul 30 01:31:47.207960 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.207869    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk"]
Jul 30 01:31:47.212540 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.212499    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk"
Jul 30 01:31:47.215237 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.215215    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-55b321\""
Jul 30 01:31:47.215341 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.215291    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-55b321-dockercfg-stzq2\""
Jul 30 01:31:47.223101 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.223077    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk"]
Jul 30 01:31:47.231745 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.231718    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr"]
Jul 30 01:31:47.235810 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.235788    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr"
Jul 30 01:31:47.260910 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.260875    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr"]
Jul 30 01:31:47.282560 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.282514    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8c06890-b1c5-4062-a793-5ad1ade8739b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk\" (UID: \"f8c06890-b1c5-4062-a793-5ad1ade8739b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk"
Jul 30 01:31:47.282560 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.282564    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvvv9\" (UniqueName: \"kubernetes.io/projected/d0e44a64-1340-44fc-a03f-cf80443d8854-kube-api-access-vvvv9\") pod \"payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr\" (UID: \"d0e44a64-1340-44fc-a03f-cf80443d8854\") " pod="openshift-ingress/payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr"
Jul 30 01:31:47.282756 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.282597    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0e44a64-1340-44fc-a03f-cf80443d8854-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr\" (UID: \"d0e44a64-1340-44fc-a03f-cf80443d8854\") " pod="openshift-ingress/payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr"
Jul 30 01:31:47.282756 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.282623    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd6z4\" (UniqueName: \"kubernetes.io/projected/f8c06890-b1c5-4062-a793-5ad1ade8739b-kube-api-access-nd6z4\") pod \"payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk\" (UID: \"f8c06890-b1c5-4062-a793-5ad1ade8739b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk"
Jul 30 01:31:47.383187 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.383153    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8c06890-b1c5-4062-a793-5ad1ade8739b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk\" (UID: \"f8c06890-b1c5-4062-a793-5ad1ade8739b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk"
Jul 30 01:31:47.383187 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.383189    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvvv9\" (UniqueName: \"kubernetes.io/projected/d0e44a64-1340-44fc-a03f-cf80443d8854-kube-api-access-vvvv9\") pod \"payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr\" (UID: \"d0e44a64-1340-44fc-a03f-cf80443d8854\") " pod="openshift-ingress/payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr"
Jul 30 01:31:47.383397 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.383223    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0e44a64-1340-44fc-a03f-cf80443d8854-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr\" (UID: \"d0e44a64-1340-44fc-a03f-cf80443d8854\") " pod="openshift-ingress/payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr"
Jul 30 01:31:47.383397 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.383250    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd6z4\" (UniqueName: \"kubernetes.io/projected/f8c06890-b1c5-4062-a793-5ad1ade8739b-kube-api-access-nd6z4\") pod \"payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk\" (UID: \"f8c06890-b1c5-4062-a793-5ad1ade8739b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk"
Jul 30 01:31:47.383969 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.383948    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0e44a64-1340-44fc-a03f-cf80443d8854-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr\" (UID: \"d0e44a64-1340-44fc-a03f-cf80443d8854\") " pod="openshift-ingress/payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr"
Jul 30 01:31:47.384018 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.383954    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8c06890-b1c5-4062-a793-5ad1ade8739b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk\" (UID: \"f8c06890-b1c5-4062-a793-5ad1ade8739b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk"
Jul 30 01:31:47.391240 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.391195    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvvv9\" (UniqueName: \"kubernetes.io/projected/d0e44a64-1340-44fc-a03f-cf80443d8854-kube-api-access-vvvv9\") pod \"payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr\" (UID: \"d0e44a64-1340-44fc-a03f-cf80443d8854\") " pod="openshift-ingress/payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr"
Jul 30 01:31:47.391387 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.391369    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd6z4\" (UniqueName: \"kubernetes.io/projected/f8c06890-b1c5-4062-a793-5ad1ade8739b-kube-api-access-nd6z4\") pod \"payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk\" (UID: \"f8c06890-b1c5-4062-a793-5ad1ade8739b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk"
Jul 30 01:31:47.524966 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.524932    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk"
Jul 30 01:31:47.556779 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.556751    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr"
Jul 30 01:31:47.883337 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.883309    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk"]
Jul 30 01:31:47.884829 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:31:47.884790    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8c06890_b1c5_4062_a793_5ad1ade8739b.slice/crio-5cc6a2b1ef49ff946aecf60d9ab5d171391fc514ead221047b07e0b1a10590a3 WatchSource:0}: Error finding container 5cc6a2b1ef49ff946aecf60d9ab5d171391fc514ead221047b07e0b1a10590a3: Status 404 returned error can't find the container with id 5cc6a2b1ef49ff946aecf60d9ab5d171391fc514ead221047b07e0b1a10590a3
Jul 30 01:31:47.917087 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.917048    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk" event={"ID":"f8c06890-b1c5-4062-a793-5ad1ade8739b","Type":"ContainerStarted","Data":"5cc6a2b1ef49ff946aecf60d9ab5d171391fc514ead221047b07e0b1a10590a3"}
Jul 30 01:31:47.918480 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.918458    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc" event={"ID":"a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2","Type":"ContainerStarted","Data":"ee796a26445648d0d81ecc34cefa03fbf0b9e7ba501508d4e2878cc44410abd0"}
Jul 30 01:31:47.918597 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.918497    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc" event={"ID":"a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2","Type":"ContainerStarted","Data":"9d70eab84c63f2ad20c1845221ac9884f90614286409674a5602dcb893cb056e"}
Jul 30 01:31:47.918597 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.918572    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc"
Jul 30 01:31:47.926824 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.926802    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr"]
Jul 30 01:31:47.935239 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:31:47.935210    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0e44a64_1340_44fc_a03f_cf80443d8854.slice/crio-1df6f810416b671b13f7cfb3373a691aa0a293b695fe90c9ba37450140949fd4 WatchSource:0}: Error finding container 1df6f810416b671b13f7cfb3373a691aa0a293b695fe90c9ba37450140949fd4: Status 404 returned error can't find the container with id 1df6f810416b671b13f7cfb3373a691aa0a293b695fe90c9ba37450140949fd4
Jul 30 01:31:47.944760 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:47.944712    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc" podStartSLOduration=1.5561425519999998 podStartE2EDuration="1.944694514s" totalImagesPullingTime="388.551962ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:31:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:31:47.94417893 +0000 UTC m=+2895.166912747" watchObservedRunningTime="2026-07-30 01:31:47.944694514 +0000 UTC m=+2895.167428336"
Jul 30 01:31:48.925696 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:48.925657    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk" event={"ID":"f8c06890-b1c5-4062-a793-5ad1ade8739b","Type":"ContainerStarted","Data":"fe0c1ae95d78981c15f2a4659c90b4d604112212dea40ef4ea4add2917436878"}
Jul 30 01:31:48.926180 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:48.925728    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk"
Jul 30 01:31:48.927326 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:48.927303    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr" event={"ID":"d0e44a64-1340-44fc-a03f-cf80443d8854","Type":"ContainerStarted","Data":"890916ab1bf669685f8a35f173a3681033d8afa9bcc3a78c1abe0471366db87c"}
Jul 30 01:31:48.927440 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:48.927344    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr" event={"ID":"d0e44a64-1340-44fc-a03f-cf80443d8854","Type":"ContainerStarted","Data":"1df6f810416b671b13f7cfb3373a691aa0a293b695fe90c9ba37450140949fd4"}
Jul 30 01:31:48.927440 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:48.927372    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr"
Jul 30 01:31:48.941618 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:48.941568    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk" podStartSLOduration=1.941553661 podStartE2EDuration="1.941553661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:31:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:31:48.940719762 +0000 UTC m=+2896.163453580" watchObservedRunningTime="2026-07-30 01:31:48.941553661 +0000 UTC m=+2896.164287478"
Jul 30 01:31:48.957495 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:48.957448    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr" podStartSLOduration=1.957435442 podStartE2EDuration="1.957435442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:31:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:31:48.956318564 +0000 UTC m=+2896.179052387" watchObservedRunningTime="2026-07-30 01:31:48.957435442 +0000 UTC m=+2896.180169259"
Jul 30 01:31:53.932772 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:53.932736    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc"
Jul 30 01:31:59.934538 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:59.934502    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk"
Jul 30 01:31:59.934920 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:31:59.934619    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr"
Jul 30 01:32:15.193693 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.193656    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f"]
Jul 30 01:32:15.196411 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.196390    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f"
Jul 30 01:32:15.199300 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.199278    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-fba93c-serving-cert\""
Jul 30 01:32:15.210734 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.210708    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f"]
Jul 30 01:32:15.317593 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.317556    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44825109-2ad0-4401-845e-0bd91f61256f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f\" (UID: \"44825109-2ad0-4401-845e-0bd91f61256f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f"
Jul 30 01:32:15.317796 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.317663    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44825109-2ad0-4401-845e-0bd91f61256f-maas-api-tls\") pod \"maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f\" (UID: \"44825109-2ad0-4401-845e-0bd91f61256f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f"
Jul 30 01:32:15.317796 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.317704    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-284sb\" (UniqueName: \"kubernetes.io/projected/44825109-2ad0-4401-845e-0bd91f61256f-kube-api-access-284sb\") pod \"maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f\" (UID: \"44825109-2ad0-4401-845e-0bd91f61256f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f"
Jul 30 01:32:15.418402 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.418367    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44825109-2ad0-4401-845e-0bd91f61256f-maas-api-tls\") pod \"maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f\" (UID: \"44825109-2ad0-4401-845e-0bd91f61256f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f"
Jul 30 01:32:15.418402 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.418403    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-284sb\" (UniqueName: \"kubernetes.io/projected/44825109-2ad0-4401-845e-0bd91f61256f-kube-api-access-284sb\") pod \"maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f\" (UID: \"44825109-2ad0-4401-845e-0bd91f61256f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f"
Jul 30 01:32:15.418666 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.418476    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44825109-2ad0-4401-845e-0bd91f61256f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f\" (UID: \"44825109-2ad0-4401-845e-0bd91f61256f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f"
Jul 30 01:32:15.419180 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.419157    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44825109-2ad0-4401-845e-0bd91f61256f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f\" (UID: \"44825109-2ad0-4401-845e-0bd91f61256f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f"
Jul 30 01:32:15.420803 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.420783    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44825109-2ad0-4401-845e-0bd91f61256f-maas-api-tls\") pod \"maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f\" (UID: \"44825109-2ad0-4401-845e-0bd91f61256f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f"
Jul 30 01:32:15.428466 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.428445    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-284sb\" (UniqueName: \"kubernetes.io/projected/44825109-2ad0-4401-845e-0bd91f61256f-kube-api-access-284sb\") pod \"maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f\" (UID: \"44825109-2ad0-4401-845e-0bd91f61256f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f"
Jul 30 01:32:15.508323 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.508298    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f"
Jul 30 01:32:15.652175 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.652154    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f"]
Jul 30 01:32:15.654293 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:32:15.654266    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44825109_2ad0_4401_845e_0bd91f61256f.slice/crio-b5dfee95316ede9ec47e8bcfaa60c3375b1243794eef77aa62a206f135c57967 WatchSource:0}: Error finding container b5dfee95316ede9ec47e8bcfaa60c3375b1243794eef77aa62a206f135c57967: Status 404 returned error can't find the container with id b5dfee95316ede9ec47e8bcfaa60c3375b1243794eef77aa62a206f135c57967
Jul 30 01:32:15.819408 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.819321    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd"]
Jul 30 01:32:15.822815 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.822796    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd"
Jul 30 01:32:15.825643 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.825616    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-fba93c-dockercfg-25vk7\""
Jul 30 01:32:15.825643 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.825635    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-fba93c\""
Jul 30 01:32:15.831833 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.831810    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd"]
Jul 30 01:32:15.839656 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.839635    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn"]
Jul 30 01:32:15.842125 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.842107    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn"
Jul 30 01:32:15.858040 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.858015    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn"]
Jul 30 01:32:15.923756 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.923712    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcn9r\" (UniqueName: \"kubernetes.io/projected/dd93670e-7cc5-4c60-be8b-606e79fa5eb0-kube-api-access-fcn9r\") pod \"payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd\" (UID: \"dd93670e-7cc5-4c60-be8b-606e79fa5eb0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd"
Jul 30 01:32:15.923756 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.923752    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjczg\" (UniqueName: \"kubernetes.io/projected/f0ceb8f3-69e9-4581-8312-57701cea8163-kube-api-access-qjczg\") pod \"payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn\" (UID: \"f0ceb8f3-69e9-4581-8312-57701cea8163\") " pod="openshift-ingress/payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn"
Jul 30 01:32:15.923998 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.923855    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd93670e-7cc5-4c60-be8b-606e79fa5eb0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd\" (UID: \"dd93670e-7cc5-4c60-be8b-606e79fa5eb0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd"
Jul 30 01:32:15.923998 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:15.923916    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0ceb8f3-69e9-4581-8312-57701cea8163-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn\" (UID: \"f0ceb8f3-69e9-4581-8312-57701cea8163\") " pod="openshift-ingress/payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn"
Jul 30 01:32:16.024772 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:16.024747    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcn9r\" (UniqueName: \"kubernetes.io/projected/dd93670e-7cc5-4c60-be8b-606e79fa5eb0-kube-api-access-fcn9r\") pod \"payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd\" (UID: \"dd93670e-7cc5-4c60-be8b-606e79fa5eb0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd"
Jul 30 01:32:16.024901 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:16.024783    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjczg\" (UniqueName: \"kubernetes.io/projected/f0ceb8f3-69e9-4581-8312-57701cea8163-kube-api-access-qjczg\") pod \"payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn\" (UID: \"f0ceb8f3-69e9-4581-8312-57701cea8163\") " pod="openshift-ingress/payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn"
Jul 30 01:32:16.024966 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:16.024931    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd93670e-7cc5-4c60-be8b-606e79fa5eb0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd\" (UID: \"dd93670e-7cc5-4c60-be8b-606e79fa5eb0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd"
Jul 30 01:32:16.025031 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:16.025013    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0ceb8f3-69e9-4581-8312-57701cea8163-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn\" (UID: \"f0ceb8f3-69e9-4581-8312-57701cea8163\") " pod="openshift-ingress/payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn"
Jul 30 01:32:16.025687 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:16.025668    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0ceb8f3-69e9-4581-8312-57701cea8163-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn\" (UID: \"f0ceb8f3-69e9-4581-8312-57701cea8163\") " pod="openshift-ingress/payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn"
Jul 30 01:32:16.025767 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:16.025675    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd93670e-7cc5-4c60-be8b-606e79fa5eb0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd\" (UID: \"dd93670e-7cc5-4c60-be8b-606e79fa5eb0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd"
Jul 30 01:32:16.032755 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:16.032733    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcn9r\" (UniqueName: \"kubernetes.io/projected/dd93670e-7cc5-4c60-be8b-606e79fa5eb0-kube-api-access-fcn9r\") pod \"payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd\" (UID: \"dd93670e-7cc5-4c60-be8b-606e79fa5eb0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd"
Jul 30 01:32:16.033066 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:16.033047    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjczg\" (UniqueName: \"kubernetes.io/projected/f0ceb8f3-69e9-4581-8312-57701cea8163-kube-api-access-qjczg\") pod \"payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn\" (UID: \"f0ceb8f3-69e9-4581-8312-57701cea8163\") " pod="openshift-ingress/payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn"
Jul 30 01:32:16.068077 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:16.068041    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f" event={"ID":"44825109-2ad0-4401-845e-0bd91f61256f","Type":"ContainerStarted","Data":"6074fb1a8e4ef366229d8d440bd96a62aa2fbd19c665fd73cb5760231e7c3cdf"}
Jul 30 01:32:16.068257 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:16.068107    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f"
Jul 30 01:32:16.068257 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:16.068125    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f" event={"ID":"44825109-2ad0-4401-845e-0bd91f61256f","Type":"ContainerStarted","Data":"b5dfee95316ede9ec47e8bcfaa60c3375b1243794eef77aa62a206f135c57967"}
Jul 30 01:32:16.087253 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:16.087137    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f" podStartSLOduration=0.764781618 podStartE2EDuration="1.087117532s" totalImagesPullingTime="322.335914ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:32:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:32:16.08357433 +0000 UTC m=+2923.306308145" watchObservedRunningTime="2026-07-30 01:32:16.087117532 +0000 UTC m=+2923.309851351"
Jul 30 01:32:16.134453 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:16.134419    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd"
Jul 30 01:32:16.154048 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:16.154021    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn"
Jul 30 01:32:16.298135 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:16.298079    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd"]
Jul 30 01:32:16.538304 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:16.538276    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn"]
Jul 30 01:32:16.540150 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:32:16.540120    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0ceb8f3_69e9_4581_8312_57701cea8163.slice/crio-06d05b869ff57297b2884bbdb37e8b884c6da8d92e6e17989d4b89970dc38770 WatchSource:0}: Error finding container 06d05b869ff57297b2884bbdb37e8b884c6da8d92e6e17989d4b89970dc38770: Status 404 returned error can't find the container with id 06d05b869ff57297b2884bbdb37e8b884c6da8d92e6e17989d4b89970dc38770
Jul 30 01:32:17.075017 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:17.074974    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn" event={"ID":"f0ceb8f3-69e9-4581-8312-57701cea8163","Type":"ContainerStarted","Data":"598925a0ed14bc3fb1bfe960b7b6ccb3ce27425d86237025a96d3ea3d4a48476"}
Jul 30 01:32:17.075222 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:17.075037    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn" event={"ID":"f0ceb8f3-69e9-4581-8312-57701cea8163","Type":"ContainerStarted","Data":"06d05b869ff57297b2884bbdb37e8b884c6da8d92e6e17989d4b89970dc38770"}
Jul 30 01:32:17.075222 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:17.075145    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn"
Jul 30 01:32:17.076503 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:17.076479    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd" event={"ID":"dd93670e-7cc5-4c60-be8b-606e79fa5eb0","Type":"ContainerStarted","Data":"8a22b288267b148ffe0dea0273e7e8a598c46108274f0e02970a1dfaad72ca29"}
Jul 30 01:32:17.076639 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:17.076517    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd" event={"ID":"dd93670e-7cc5-4c60-be8b-606e79fa5eb0","Type":"ContainerStarted","Data":"5ac2f6d9d48b5182dd8382e7e577721c718a7951798bbff383223ea98a244e7e"}
Jul 30 01:32:17.076639 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:17.076568    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd"
Jul 30 01:32:17.098704 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:17.098644    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn" podStartSLOduration=2.098626065 podStartE2EDuration="2.098626065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:32:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:32:17.091928058 +0000 UTC m=+2924.314661882" watchObservedRunningTime="2026-07-30 01:32:17.098626065 +0000 UTC m=+2924.321359884"
Jul 30 01:32:17.110901 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:17.110848    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd" podStartSLOduration=2.110832091 podStartE2EDuration="2.110832091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:32:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:32:17.110069281 +0000 UTC m=+2924.332803098" watchObservedRunningTime="2026-07-30 01:32:17.110832091 +0000 UTC m=+2924.333565909"
Jul 30 01:32:22.082018 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:22.081991    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f"
Jul 30 01:32:28.083443 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:28.083410    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd"
Jul 30 01:32:28.083884 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:32:28.083545    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn"
Jul 30 01:33:33.445864 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:33:33.445838    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 01:33:33.451419 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:33:33.451396    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 01:38:33.482371 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:38:33.482340    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 01:38:33.488316 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:38:33.488287    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 01:43:13.733022 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:13.732974    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf"]
Jul 30 01:43:13.736586 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:13.736561    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf"
Jul 30 01:43:13.739084 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:13.739063    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-09b0d2-serving-cert\""
Jul 30 01:43:13.746228 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:13.746200    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf"]
Jul 30 01:43:13.864779 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:13.864748    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ce4a97c-37f4-43f4-aecf-507f685c67b4-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf\" (UID: \"0ce4a97c-37f4-43f4-aecf-507f685c67b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf"
Jul 30 01:43:13.864958 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:13.864827    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wmdc\" (UniqueName: \"kubernetes.io/projected/0ce4a97c-37f4-43f4-aecf-507f685c67b4-kube-api-access-4wmdc\") pod \"maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf\" (UID: \"0ce4a97c-37f4-43f4-aecf-507f685c67b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf"
Jul 30 01:43:13.864958 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:13.864865    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ce4a97c-37f4-43f4-aecf-507f685c67b4-maas-api-tls\") pod \"maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf\" (UID: \"0ce4a97c-37f4-43f4-aecf-507f685c67b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf"
Jul 30 01:43:13.965827 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:13.965780    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ce4a97c-37f4-43f4-aecf-507f685c67b4-maas-api-tls\") pod \"maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf\" (UID: \"0ce4a97c-37f4-43f4-aecf-507f685c67b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf"
Jul 30 01:43:13.966041 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:13.965849    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ce4a97c-37f4-43f4-aecf-507f685c67b4-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf\" (UID: \"0ce4a97c-37f4-43f4-aecf-507f685c67b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf"
Jul 30 01:43:13.966041 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:43:13.965934    2546 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-serving-cert: secret "maas-api-e2e-ipp-a-09b0d2-serving-cert" not found
Jul 30 01:43:13.966041 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:43:13.966002    2546 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ce4a97c-37f4-43f4-aecf-507f685c67b4-maas-api-tls podName:0ce4a97c-37f4-43f4-aecf-507f685c67b4 nodeName:}" failed. No retries permitted until 2026-07-30 01:43:14.465985212 +0000 UTC m=+3581.688719012 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0ce4a97c-37f4-43f4-aecf-507f685c67b4-maas-api-tls") pod "maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf" (UID: "0ce4a97c-37f4-43f4-aecf-507f685c67b4") : secret "maas-api-e2e-ipp-a-09b0d2-serving-cert" not found
Jul 30 01:43:13.966041 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:13.965947    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wmdc\" (UniqueName: \"kubernetes.io/projected/0ce4a97c-37f4-43f4-aecf-507f685c67b4-kube-api-access-4wmdc\") pod \"maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf\" (UID: \"0ce4a97c-37f4-43f4-aecf-507f685c67b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf"
Jul 30 01:43:13.966497 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:13.966476    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ce4a97c-37f4-43f4-aecf-507f685c67b4-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf\" (UID: \"0ce4a97c-37f4-43f4-aecf-507f685c67b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf"
Jul 30 01:43:13.974340 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:13.974317    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wmdc\" (UniqueName: \"kubernetes.io/projected/0ce4a97c-37f4-43f4-aecf-507f685c67b4-kube-api-access-4wmdc\") pod \"maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf\" (UID: \"0ce4a97c-37f4-43f4-aecf-507f685c67b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf"
Jul 30 01:43:14.371855 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.371818    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz"]
Jul 30 01:43:14.375411 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.375390    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz"
Jul 30 01:43:14.380940 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.380919    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-09b0d2-dockercfg-h7c4t\""
Jul 30 01:43:14.381056 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.380981    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-09b0d2\""
Jul 30 01:43:14.387173 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.387145    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz"]
Jul 30 01:43:14.401457 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.401415    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72"]
Jul 30 01:43:14.405008 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.404986    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72"
Jul 30 01:43:14.413980 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.413954    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72"]
Jul 30 01:43:14.471118 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.471081    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51b2447f-118b-4b88-af9e-4b1e36c7293e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz\" (UID: \"51b2447f-118b-4b88-af9e-4b1e36c7293e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz"
Jul 30 01:43:14.471313 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.471169    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ce4a97c-37f4-43f4-aecf-507f685c67b4-maas-api-tls\") pod \"maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf\" (UID: \"0ce4a97c-37f4-43f4-aecf-507f685c67b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf"
Jul 30 01:43:14.471313 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.471218    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnkvd\" (UniqueName: \"kubernetes.io/projected/51b2447f-118b-4b88-af9e-4b1e36c7293e-kube-api-access-fnkvd\") pod \"payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz\" (UID: \"51b2447f-118b-4b88-af9e-4b1e36c7293e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz"
Jul 30 01:43:14.473812 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.473787    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ce4a97c-37f4-43f4-aecf-507f685c67b4-maas-api-tls\") pod \"maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf\" (UID: \"0ce4a97c-37f4-43f4-aecf-507f685c67b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf"
Jul 30 01:43:14.572341 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.572299    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51b2447f-118b-4b88-af9e-4b1e36c7293e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz\" (UID: \"51b2447f-118b-4b88-af9e-4b1e36c7293e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz"
Jul 30 01:43:14.572341 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.572340    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2eb2c373-ba22-49fa-9968-3184a6272bf9-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72\" (UID: \"2eb2c373-ba22-49fa-9968-3184a6272bf9\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72"
Jul 30 01:43:14.572649 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.572421    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2mtq\" (UniqueName: \"kubernetes.io/projected/2eb2c373-ba22-49fa-9968-3184a6272bf9-kube-api-access-q2mtq\") pod \"payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72\" (UID: \"2eb2c373-ba22-49fa-9968-3184a6272bf9\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72"
Jul 30 01:43:14.572649 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.572513    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnkvd\" (UniqueName: \"kubernetes.io/projected/51b2447f-118b-4b88-af9e-4b1e36c7293e-kube-api-access-fnkvd\") pod \"payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz\" (UID: \"51b2447f-118b-4b88-af9e-4b1e36c7293e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz"
Jul 30 01:43:14.573047 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.573028    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51b2447f-118b-4b88-af9e-4b1e36c7293e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz\" (UID: \"51b2447f-118b-4b88-af9e-4b1e36c7293e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz"
Jul 30 01:43:14.580095 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.580073    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnkvd\" (UniqueName: \"kubernetes.io/projected/51b2447f-118b-4b88-af9e-4b1e36c7293e-kube-api-access-fnkvd\") pod \"payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz\" (UID: \"51b2447f-118b-4b88-af9e-4b1e36c7293e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz"
Jul 30 01:43:14.649512 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.649416    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf"
Jul 30 01:43:14.673374 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.673340    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2eb2c373-ba22-49fa-9968-3184a6272bf9-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72\" (UID: \"2eb2c373-ba22-49fa-9968-3184a6272bf9\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72"
Jul 30 01:43:14.673510 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.673450    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2mtq\" (UniqueName: \"kubernetes.io/projected/2eb2c373-ba22-49fa-9968-3184a6272bf9-kube-api-access-q2mtq\") pod \"payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72\" (UID: \"2eb2c373-ba22-49fa-9968-3184a6272bf9\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72"
Jul 30 01:43:14.674295 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.674272    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2eb2c373-ba22-49fa-9968-3184a6272bf9-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72\" (UID: \"2eb2c373-ba22-49fa-9968-3184a6272bf9\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72"
Jul 30 01:43:14.681087 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.681064    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2mtq\" (UniqueName: \"kubernetes.io/projected/2eb2c373-ba22-49fa-9968-3184a6272bf9-kube-api-access-q2mtq\") pod \"payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72\" (UID: \"2eb2c373-ba22-49fa-9968-3184a6272bf9\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72"
Jul 30 01:43:14.688062 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.688019    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz"
Jul 30 01:43:14.718113 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:14.718076    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72"
Jul 30 01:43:15.029185 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:15.029157    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf"]
Jul 30 01:43:15.030796 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:43:15.030770    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ce4a97c_37f4_43f4_aecf_507f685c67b4.slice/crio-876bcc99e52799133f15e3be7812af3337570562ee3187595b3a623cf27824e8 WatchSource:0}: Error finding container 876bcc99e52799133f15e3be7812af3337570562ee3187595b3a623cf27824e8: Status 404 returned error can't find the container with id 876bcc99e52799133f15e3be7812af3337570562ee3187595b3a623cf27824e8
Jul 30 01:43:15.032151 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:15.032129    2546 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 01:43:15.069048 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:15.069020    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz"]
Jul 30 01:43:15.070928 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:43:15.070901    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51b2447f_118b_4b88_af9e_4b1e36c7293e.slice/crio-d7b78233e3525b24c404397df58d6b17ad716b351242271e643e86e1696b8924 WatchSource:0}: Error finding container d7b78233e3525b24c404397df58d6b17ad716b351242271e643e86e1696b8924: Status 404 returned error can't find the container with id d7b78233e3525b24c404397df58d6b17ad716b351242271e643e86e1696b8924
Jul 30 01:43:15.105650 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:15.105623    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72"]
Jul 30 01:43:15.108789 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:43:15.108766    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2eb2c373_ba22_49fa_9968_3184a6272bf9.slice/crio-556508e767faf6b683c7c1618ec2e23b297677b340358b408a22950a20f58c97 WatchSource:0}: Error finding container 556508e767faf6b683c7c1618ec2e23b297677b340358b408a22950a20f58c97: Status 404 returned error can't find the container with id 556508e767faf6b683c7c1618ec2e23b297677b340358b408a22950a20f58c97
Jul 30 01:43:15.563709 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:15.563677    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72" event={"ID":"2eb2c373-ba22-49fa-9968-3184a6272bf9","Type":"ContainerStarted","Data":"787e308d40ca387a73802e1e684f504c10ce34fbf203d76946f4bcbbeae827ca"}
Jul 30 01:43:15.563930 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:15.563743    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72"
Jul 30 01:43:15.563930 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:15.563769    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72" event={"ID":"2eb2c373-ba22-49fa-9968-3184a6272bf9","Type":"ContainerStarted","Data":"556508e767faf6b683c7c1618ec2e23b297677b340358b408a22950a20f58c97"}
Jul 30 01:43:15.564916 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:15.564892    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf" event={"ID":"0ce4a97c-37f4-43f4-aecf-507f685c67b4","Type":"ContainerStarted","Data":"876bcc99e52799133f15e3be7812af3337570562ee3187595b3a623cf27824e8"}
Jul 30 01:43:15.566241 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:15.566219    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz" event={"ID":"51b2447f-118b-4b88-af9e-4b1e36c7293e","Type":"ContainerStarted","Data":"73c9f5715c3d4612688bc46494514b2ddb7ff82bf5be92f9c4c5084b77c17fb2"}
Jul 30 01:43:15.566352 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:15.566261    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz" event={"ID":"51b2447f-118b-4b88-af9e-4b1e36c7293e","Type":"ContainerStarted","Data":"d7b78233e3525b24c404397df58d6b17ad716b351242271e643e86e1696b8924"}
Jul 30 01:43:15.566352 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:15.566289    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz"
Jul 30 01:43:15.580678 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:15.580632    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72" podStartSLOduration=1.580618106 podStartE2EDuration="1.580618106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:43:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:43:15.578439451 +0000 UTC m=+3582.801173268" watchObservedRunningTime="2026-07-30 01:43:15.580618106 +0000 UTC m=+3582.803351959"
Jul 30 01:43:15.595438 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:15.595385    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz" podStartSLOduration=1.5953716629999999 podStartE2EDuration="1.595371663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:43:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:43:15.592552074 +0000 UTC m=+3582.815285889" watchObservedRunningTime="2026-07-30 01:43:15.595371663 +0000 UTC m=+3582.818105478"
Jul 30 01:43:16.574910 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:16.574871    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf" event={"ID":"0ce4a97c-37f4-43f4-aecf-507f685c67b4","Type":"ContainerStarted","Data":"73950c40e729e014f554d8c30f1b1ccca5574300ada5adaca0e6a3f6b2b858f3"}
Jul 30 01:43:16.575347 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:16.575123    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf"
Jul 30 01:43:16.593555 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:16.593496    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf" podStartSLOduration=3.019559156 podStartE2EDuration="3.593478743s" totalImagesPullingTime="573.919587ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:43:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:43:16.59176735 +0000 UTC m=+3583.814501188" watchObservedRunningTime="2026-07-30 01:43:16.593478743 +0000 UTC m=+3583.816212579"
Jul 30 01:43:22.586001 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:22.585971    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf"
Jul 30 01:43:26.577567 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:26.577470    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72"
Jul 30 01:43:26.577996 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:26.577854    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz"
Jul 30 01:43:33.519967 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:33.519937    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 01:43:33.527430 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:33.527409    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 01:43:43.509447 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:43.509391    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b"]
Jul 30 01:43:43.514013 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:43.513990    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b"
Jul 30 01:43:43.516860 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:43.516828    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-b64c39-dockercfg-p99xt\""
Jul 30 01:43:43.517003 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:43.516958    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-b64c39\""
Jul 30 01:43:43.522031 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:43.522002    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b"]
Jul 30 01:43:43.552546 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:43.552480    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wb5h\" (UniqueName: \"kubernetes.io/projected/534fb707-c22c-41d9-90bb-ad4dd671e93c-kube-api-access-5wb5h\") pod \"payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b\" (UID: \"534fb707-c22c-41d9-90bb-ad4dd671e93c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b"
Jul 30 01:43:43.552693 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:43.552607    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/534fb707-c22c-41d9-90bb-ad4dd671e93c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b\" (UID: \"534fb707-c22c-41d9-90bb-ad4dd671e93c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b"
Jul 30 01:43:43.653958 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:43.653920    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wb5h\" (UniqueName: \"kubernetes.io/projected/534fb707-c22c-41d9-90bb-ad4dd671e93c-kube-api-access-5wb5h\") pod \"payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b\" (UID: \"534fb707-c22c-41d9-90bb-ad4dd671e93c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b"
Jul 30 01:43:43.654141 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:43.653975    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/534fb707-c22c-41d9-90bb-ad4dd671e93c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b\" (UID: \"534fb707-c22c-41d9-90bb-ad4dd671e93c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b"
Jul 30 01:43:43.654754 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:43.654732    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/534fb707-c22c-41d9-90bb-ad4dd671e93c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b\" (UID: \"534fb707-c22c-41d9-90bb-ad4dd671e93c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b"
Jul 30 01:43:43.662381 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:43.662354    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wb5h\" (UniqueName: \"kubernetes.io/projected/534fb707-c22c-41d9-90bb-ad4dd671e93c-kube-api-access-5wb5h\") pod \"payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b\" (UID: \"534fb707-c22c-41d9-90bb-ad4dd671e93c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b"
Jul 30 01:43:43.827704 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:43.827619    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b"
Jul 30 01:43:43.975429 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:43.975404    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b"]
Jul 30 01:43:43.978389 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:43:43.978351    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod534fb707_c22c_41d9_90bb_ad4dd671e93c.slice/crio-b0aae7d5475db7dad31e8e232d4fb1cfbe74f4b0e8123c41c0f0e6bab180deba WatchSource:0}: Error finding container b0aae7d5475db7dad31e8e232d4fb1cfbe74f4b0e8123c41c0f0e6bab180deba: Status 404 returned error can't find the container with id b0aae7d5475db7dad31e8e232d4fb1cfbe74f4b0e8123c41c0f0e6bab180deba
Jul 30 01:43:44.731446 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:44.731401    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b" event={"ID":"534fb707-c22c-41d9-90bb-ad4dd671e93c","Type":"ContainerStarted","Data":"0210bb16186a1c17777bde33e293ae349e0aeffe807e5b22eb65affb7cdd2533"}
Jul 30 01:43:44.731954 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:44.731465    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b" event={"ID":"534fb707-c22c-41d9-90bb-ad4dd671e93c","Type":"ContainerStarted","Data":"b0aae7d5475db7dad31e8e232d4fb1cfbe74f4b0e8123c41c0f0e6bab180deba"}
Jul 30 01:43:44.731954 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:44.731617    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b"
Jul 30 01:43:44.747281 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:44.747229    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b" podStartSLOduration=1.7472135610000001 podStartE2EDuration="1.747213561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:43:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:43:44.746410992 +0000 UTC m=+3611.969144812" watchObservedRunningTime="2026-07-30 01:43:44.747213561 +0000 UTC m=+3611.969947379"
Jul 30 01:43:55.739409 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:43:55.739376    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b"
Jul 30 01:44:13.604780 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:44:13.604749    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-5692l_4c08d630-9e0a-412c-8531-968e388ff89e/payload-processing/0.log"
Jul 30 01:44:13.712568 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:44:13.712532    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b_534fb707-c22c-41d9-90bb-ad4dd671e93c/payload-processing/0.log"
Jul 30 01:45:21.147126 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:45:21.147095    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72_2eb2c373-ba22-49fa-9968-3184a6272bf9/payload-processing/0.log"
Jul 30 01:45:21.266175 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:45:21.266144    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-5692l_4c08d630-9e0a-412c-8531-968e388ff89e/payload-processing/0.log"
Jul 30 01:45:21.389892 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:45:21.389858    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b_534fb707-c22c-41d9-90bb-ad4dd671e93c/payload-processing/0.log"
Jul 30 01:46:22.049103 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.049063    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf"]
Jul 30 01:46:22.049672 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.049323    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf" podUID="0ce4a97c-37f4-43f4-aecf-507f685c67b4" containerName="maas-api" containerID="cri-o://73950c40e729e014f554d8c30f1b1ccca5574300ada5adaca0e6a3f6b2b858f3" gracePeriod=30
Jul 30 01:46:22.112629 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.112585    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72"]
Jul 30 01:46:22.112856 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.112827    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72" podUID="2eb2c373-ba22-49fa-9968-3184a6272bf9" containerName="payload-processing" containerID="cri-o://787e308d40ca387a73802e1e684f504c10ce34fbf203d76946f4bcbbeae827ca" gracePeriod=30
Jul 30 01:46:22.116726 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.116695    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz"]
Jul 30 01:46:22.117104 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.117002    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz" podUID="51b2447f-118b-4b88-af9e-4b1e36c7293e" containerName="payload-pre-processing" containerID="cri-o://73c9f5715c3d4612688bc46494514b2ddb7ff82bf5be92f9c4c5084b77c17fb2" gracePeriod=30
Jul 30 01:46:22.383325 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.383297    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf"
Jul 30 01:46:22.430488 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.430463    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz"
Jul 30 01:46:22.443204 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.443177    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72"
Jul 30 01:46:22.511463 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.511421    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ce4a97c-37f4-43f4-aecf-507f685c67b4-maas-api-tls\") pod \"0ce4a97c-37f4-43f4-aecf-507f685c67b4\" (UID: \"0ce4a97c-37f4-43f4-aecf-507f685c67b4\") "
Jul 30 01:46:22.511685 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.511494    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51b2447f-118b-4b88-af9e-4b1e36c7293e-plugins-config-volume\") pod \"51b2447f-118b-4b88-af9e-4b1e36c7293e\" (UID: \"51b2447f-118b-4b88-af9e-4b1e36c7293e\") "
Jul 30 01:46:22.511685 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.511551    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnkvd\" (UniqueName: \"kubernetes.io/projected/51b2447f-118b-4b88-af9e-4b1e36c7293e-kube-api-access-fnkvd\") pod \"51b2447f-118b-4b88-af9e-4b1e36c7293e\" (UID: \"51b2447f-118b-4b88-af9e-4b1e36c7293e\") "
Jul 30 01:46:22.511685 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.511595    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ce4a97c-37f4-43f4-aecf-507f685c67b4-openshift-service-ca\") pod \"0ce4a97c-37f4-43f4-aecf-507f685c67b4\" (UID: \"0ce4a97c-37f4-43f4-aecf-507f685c67b4\") "
Jul 30 01:46:22.511685 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.511640    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4wmdc\" (UniqueName: \"kubernetes.io/projected/0ce4a97c-37f4-43f4-aecf-507f685c67b4-kube-api-access-4wmdc\") pod \"0ce4a97c-37f4-43f4-aecf-507f685c67b4\" (UID: \"0ce4a97c-37f4-43f4-aecf-507f685c67b4\") "
Jul 30 01:46:22.512292 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.511907    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51b2447f-118b-4b88-af9e-4b1e36c7293e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "51b2447f-118b-4b88-af9e-4b1e36c7293e" (UID: "51b2447f-118b-4b88-af9e-4b1e36c7293e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:46:22.512292 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.512011    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51b2447f-118b-4b88-af9e-4b1e36c7293e-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:46:22.512292 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.512249    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ce4a97c-37f4-43f4-aecf-507f685c67b4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0ce4a97c-37f4-43f4-aecf-507f685c67b4" (UID: "0ce4a97c-37f4-43f4-aecf-507f685c67b4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:46:22.513937 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.513912    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ce4a97c-37f4-43f4-aecf-507f685c67b4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0ce4a97c-37f4-43f4-aecf-507f685c67b4" (UID: "0ce4a97c-37f4-43f4-aecf-507f685c67b4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:46:22.514078 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.513962    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ce4a97c-37f4-43f4-aecf-507f685c67b4-kube-api-access-4wmdc" (OuterVolumeSpecName: "kube-api-access-4wmdc") pod "0ce4a97c-37f4-43f4-aecf-507f685c67b4" (UID: "0ce4a97c-37f4-43f4-aecf-507f685c67b4"). InnerVolumeSpecName "kube-api-access-4wmdc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:46:22.514078 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.513986    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51b2447f-118b-4b88-af9e-4b1e36c7293e-kube-api-access-fnkvd" (OuterVolumeSpecName: "kube-api-access-fnkvd") pod "51b2447f-118b-4b88-af9e-4b1e36c7293e" (UID: "51b2447f-118b-4b88-af9e-4b1e36c7293e"). InnerVolumeSpecName "kube-api-access-fnkvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:46:22.608514 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.608432    2546 generic.go:356] "Generic (PLEG): container finished" podID="51b2447f-118b-4b88-af9e-4b1e36c7293e" containerID="73c9f5715c3d4612688bc46494514b2ddb7ff82bf5be92f9c4c5084b77c17fb2" exitCode=0
Jul 30 01:46:22.608514 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.608477    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz"
Jul 30 01:46:22.608732 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.608517    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz" event={"ID":"51b2447f-118b-4b88-af9e-4b1e36c7293e","Type":"ContainerDied","Data":"73c9f5715c3d4612688bc46494514b2ddb7ff82bf5be92f9c4c5084b77c17fb2"}
Jul 30 01:46:22.608732 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.608599    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz" event={"ID":"51b2447f-118b-4b88-af9e-4b1e36c7293e","Type":"ContainerDied","Data":"d7b78233e3525b24c404397df58d6b17ad716b351242271e643e86e1696b8924"}
Jul 30 01:46:22.608732 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.608628    2546 scope.go:117] "RemoveContainer" containerID="73c9f5715c3d4612688bc46494514b2ddb7ff82bf5be92f9c4c5084b77c17fb2"
Jul 30 01:46:22.609817 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.609799    2546 generic.go:356] "Generic (PLEG): container finished" podID="2eb2c373-ba22-49fa-9968-3184a6272bf9" containerID="787e308d40ca387a73802e1e684f504c10ce34fbf203d76946f4bcbbeae827ca" exitCode=0
Jul 30 01:46:22.609906 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.609843    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72"
Jul 30 01:46:22.609906 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.609885    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72" event={"ID":"2eb2c373-ba22-49fa-9968-3184a6272bf9","Type":"ContainerDied","Data":"787e308d40ca387a73802e1e684f504c10ce34fbf203d76946f4bcbbeae827ca"}
Jul 30 01:46:22.610019 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.609956    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72" event={"ID":"2eb2c373-ba22-49fa-9968-3184a6272bf9","Type":"ContainerDied","Data":"556508e767faf6b683c7c1618ec2e23b297677b340358b408a22950a20f58c97"}
Jul 30 01:46:22.611240 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.611219    2546 generic.go:356] "Generic (PLEG): container finished" podID="0ce4a97c-37f4-43f4-aecf-507f685c67b4" containerID="73950c40e729e014f554d8c30f1b1ccca5574300ada5adaca0e6a3f6b2b858f3" exitCode=0
Jul 30 01:46:22.611338 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.611256    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf"
Jul 30 01:46:22.611338 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.611267    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf" event={"ID":"0ce4a97c-37f4-43f4-aecf-507f685c67b4","Type":"ContainerDied","Data":"73950c40e729e014f554d8c30f1b1ccca5574300ada5adaca0e6a3f6b2b858f3"}
Jul 30 01:46:22.611338 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.611318    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf" event={"ID":"0ce4a97c-37f4-43f4-aecf-507f685c67b4","Type":"ContainerDied","Data":"876bcc99e52799133f15e3be7812af3337570562ee3187595b3a623cf27824e8"}
Jul 30 01:46:22.612317 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.612271    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2mtq\" (UniqueName: \"kubernetes.io/projected/2eb2c373-ba22-49fa-9968-3184a6272bf9-kube-api-access-q2mtq\") pod \"2eb2c373-ba22-49fa-9968-3184a6272bf9\" (UID: \"2eb2c373-ba22-49fa-9968-3184a6272bf9\") "
Jul 30 01:46:22.612434 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.612316    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2eb2c373-ba22-49fa-9968-3184a6272bf9-plugins-config-volume\") pod \"2eb2c373-ba22-49fa-9968-3184a6272bf9\" (UID: \"2eb2c373-ba22-49fa-9968-3184a6272bf9\") "
Jul 30 01:46:22.612643 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.612623    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ce4a97c-37f4-43f4-aecf-507f685c67b4-openshift-service-ca\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:46:22.612736 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.612650    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wmdc\" (UniqueName: \"kubernetes.io/projected/0ce4a97c-37f4-43f4-aecf-507f685c67b4-kube-api-access-4wmdc\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:46:22.612736 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.612667    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ce4a97c-37f4-43f4-aecf-507f685c67b4-maas-api-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:46:22.612736 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.612681    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnkvd\" (UniqueName: \"kubernetes.io/projected/51b2447f-118b-4b88-af9e-4b1e36c7293e-kube-api-access-fnkvd\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:46:22.612978 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.612773    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2eb2c373-ba22-49fa-9968-3184a6272bf9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2eb2c373-ba22-49fa-9968-3184a6272bf9" (UID: "2eb2c373-ba22-49fa-9968-3184a6272bf9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:46:22.614968 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.614943    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2eb2c373-ba22-49fa-9968-3184a6272bf9-kube-api-access-q2mtq" (OuterVolumeSpecName: "kube-api-access-q2mtq") pod "2eb2c373-ba22-49fa-9968-3184a6272bf9" (UID: "2eb2c373-ba22-49fa-9968-3184a6272bf9"). InnerVolumeSpecName "kube-api-access-q2mtq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:46:22.620261 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.620239    2546 scope.go:117] "RemoveContainer" containerID="73c9f5715c3d4612688bc46494514b2ddb7ff82bf5be92f9c4c5084b77c17fb2"
Jul 30 01:46:22.620584 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:46:22.620564    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73c9f5715c3d4612688bc46494514b2ddb7ff82bf5be92f9c4c5084b77c17fb2\": container with ID starting with 73c9f5715c3d4612688bc46494514b2ddb7ff82bf5be92f9c4c5084b77c17fb2 not found: ID does not exist" containerID="73c9f5715c3d4612688bc46494514b2ddb7ff82bf5be92f9c4c5084b77c17fb2"
Jul 30 01:46:22.620669 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.620592    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73c9f5715c3d4612688bc46494514b2ddb7ff82bf5be92f9c4c5084b77c17fb2"} err="failed to get container status \"73c9f5715c3d4612688bc46494514b2ddb7ff82bf5be92f9c4c5084b77c17fb2\": rpc error: code = NotFound desc = could not find container \"73c9f5715c3d4612688bc46494514b2ddb7ff82bf5be92f9c4c5084b77c17fb2\": container with ID starting with 73c9f5715c3d4612688bc46494514b2ddb7ff82bf5be92f9c4c5084b77c17fb2 not found: ID does not exist"
Jul 30 01:46:22.620669 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.620610    2546 scope.go:117] "RemoveContainer" containerID="787e308d40ca387a73802e1e684f504c10ce34fbf203d76946f4bcbbeae827ca"
Jul 30 01:46:22.629557 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.629540    2546 scope.go:117] "RemoveContainer" containerID="787e308d40ca387a73802e1e684f504c10ce34fbf203d76946f4bcbbeae827ca"
Jul 30 01:46:22.630172 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:46:22.629820    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"787e308d40ca387a73802e1e684f504c10ce34fbf203d76946f4bcbbeae827ca\": container with ID starting with 787e308d40ca387a73802e1e684f504c10ce34fbf203d76946f4bcbbeae827ca not found: ID does not exist" containerID="787e308d40ca387a73802e1e684f504c10ce34fbf203d76946f4bcbbeae827ca"
Jul 30 01:46:22.630172 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.629865    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"787e308d40ca387a73802e1e684f504c10ce34fbf203d76946f4bcbbeae827ca"} err="failed to get container status \"787e308d40ca387a73802e1e684f504c10ce34fbf203d76946f4bcbbeae827ca\": rpc error: code = NotFound desc = could not find container \"787e308d40ca387a73802e1e684f504c10ce34fbf203d76946f4bcbbeae827ca\": container with ID starting with 787e308d40ca387a73802e1e684f504c10ce34fbf203d76946f4bcbbeae827ca not found: ID does not exist"
Jul 30 01:46:22.630172 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.629887    2546 scope.go:117] "RemoveContainer" containerID="73950c40e729e014f554d8c30f1b1ccca5574300ada5adaca0e6a3f6b2b858f3"
Jul 30 01:46:22.632638 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.632618    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz"]
Jul 30 01:46:22.638292 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.638270    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-09b0d2-96bc9d556-p64vz"]
Jul 30 01:46:22.638804 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.638778    2546 scope.go:117] "RemoveContainer" containerID="73950c40e729e014f554d8c30f1b1ccca5574300ada5adaca0e6a3f6b2b858f3"
Jul 30 01:46:22.639067 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:46:22.639041    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73950c40e729e014f554d8c30f1b1ccca5574300ada5adaca0e6a3f6b2b858f3\": container with ID starting with 73950c40e729e014f554d8c30f1b1ccca5574300ada5adaca0e6a3f6b2b858f3 not found: ID does not exist" containerID="73950c40e729e014f554d8c30f1b1ccca5574300ada5adaca0e6a3f6b2b858f3"
Jul 30 01:46:22.639143 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.639075    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73950c40e729e014f554d8c30f1b1ccca5574300ada5adaca0e6a3f6b2b858f3"} err="failed to get container status \"73950c40e729e014f554d8c30f1b1ccca5574300ada5adaca0e6a3f6b2b858f3\": rpc error: code = NotFound desc = could not find container \"73950c40e729e014f554d8c30f1b1ccca5574300ada5adaca0e6a3f6b2b858f3\": container with ID starting with 73950c40e729e014f554d8c30f1b1ccca5574300ada5adaca0e6a3f6b2b858f3 not found: ID does not exist"
Jul 30 01:46:22.648768 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.648743    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf"]
Jul 30 01:46:22.650615 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.650595    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-09b0d2-7d68fcdb9d-9fbhf"]
Jul 30 01:46:22.713951 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.713914    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2mtq\" (UniqueName: \"kubernetes.io/projected/2eb2c373-ba22-49fa-9968-3184a6272bf9-kube-api-access-q2mtq\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:46:22.713951 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.713946    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2eb2c373-ba22-49fa-9968-3184a6272bf9-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:46:22.943918 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.943880    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72"]
Jul 30 01:46:22.946121 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:22.946098    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-09b0d2-5987f97588-qdf72"]
Jul 30 01:46:23.358607 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:23.358542    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ce4a97c-37f4-43f4-aecf-507f685c67b4" path="/var/lib/kubelet/pods/0ce4a97c-37f4-43f4-aecf-507f685c67b4/volumes"
Jul 30 01:46:23.358972 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:23.358935    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2eb2c373-ba22-49fa-9968-3184a6272bf9" path="/var/lib/kubelet/pods/2eb2c373-ba22-49fa-9968-3184a6272bf9/volumes"
Jul 30 01:46:23.359248 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:46:23.359235    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51b2447f-118b-4b88-af9e-4b1e36c7293e" path="/var/lib/kubelet/pods/51b2447f-118b-4b88-af9e-4b1e36c7293e/volumes"
Jul 30 01:47:35.296404 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:35.296367    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b"]
Jul 30 01:47:35.297030 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:35.296632    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b" podUID="534fb707-c22c-41d9-90bb-ad4dd671e93c" containerName="payload-processing" containerID="cri-o://0210bb16186a1c17777bde33e293ae349e0aeffe807e5b22eb65affb7cdd2533" gracePeriod=30
Jul 30 01:47:35.550769 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:35.550709    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b"
Jul 30 01:47:35.659093 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:35.659050    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wb5h\" (UniqueName: \"kubernetes.io/projected/534fb707-c22c-41d9-90bb-ad4dd671e93c-kube-api-access-5wb5h\") pod \"534fb707-c22c-41d9-90bb-ad4dd671e93c\" (UID: \"534fb707-c22c-41d9-90bb-ad4dd671e93c\") "
Jul 30 01:47:35.659276 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:35.659150    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/534fb707-c22c-41d9-90bb-ad4dd671e93c-plugins-config-volume\") pod \"534fb707-c22c-41d9-90bb-ad4dd671e93c\" (UID: \"534fb707-c22c-41d9-90bb-ad4dd671e93c\") "
Jul 30 01:47:35.659484 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:35.659452    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/534fb707-c22c-41d9-90bb-ad4dd671e93c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "534fb707-c22c-41d9-90bb-ad4dd671e93c" (UID: "534fb707-c22c-41d9-90bb-ad4dd671e93c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:47:35.661084 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:35.661064    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/534fb707-c22c-41d9-90bb-ad4dd671e93c-kube-api-access-5wb5h" (OuterVolumeSpecName: "kube-api-access-5wb5h") pod "534fb707-c22c-41d9-90bb-ad4dd671e93c" (UID: "534fb707-c22c-41d9-90bb-ad4dd671e93c"). InnerVolumeSpecName "kube-api-access-5wb5h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:47:35.760299 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:35.760260    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/534fb707-c22c-41d9-90bb-ad4dd671e93c-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:47:35.760299 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:35.760293    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wb5h\" (UniqueName: \"kubernetes.io/projected/534fb707-c22c-41d9-90bb-ad4dd671e93c-kube-api-access-5wb5h\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:47:35.996240 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:35.996205    2546 generic.go:356] "Generic (PLEG): container finished" podID="534fb707-c22c-41d9-90bb-ad4dd671e93c" containerID="0210bb16186a1c17777bde33e293ae349e0aeffe807e5b22eb65affb7cdd2533" exitCode=0
Jul 30 01:47:35.996240 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:35.996252    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b"
Jul 30 01:47:35.996510 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:35.996285    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b" event={"ID":"534fb707-c22c-41d9-90bb-ad4dd671e93c","Type":"ContainerDied","Data":"0210bb16186a1c17777bde33e293ae349e0aeffe807e5b22eb65affb7cdd2533"}
Jul 30 01:47:35.996510 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:35.996346    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b" event={"ID":"534fb707-c22c-41d9-90bb-ad4dd671e93c","Type":"ContainerDied","Data":"b0aae7d5475db7dad31e8e232d4fb1cfbe74f4b0e8123c41c0f0e6bab180deba"}
Jul 30 01:47:35.996510 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:35.996372    2546 scope.go:117] "RemoveContainer" containerID="0210bb16186a1c17777bde33e293ae349e0aeffe807e5b22eb65affb7cdd2533"
Jul 30 01:47:36.006109 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:36.006083    2546 scope.go:117] "RemoveContainer" containerID="0210bb16186a1c17777bde33e293ae349e0aeffe807e5b22eb65affb7cdd2533"
Jul 30 01:47:36.006386 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:47:36.006367    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0210bb16186a1c17777bde33e293ae349e0aeffe807e5b22eb65affb7cdd2533\": container with ID starting with 0210bb16186a1c17777bde33e293ae349e0aeffe807e5b22eb65affb7cdd2533 not found: ID does not exist" containerID="0210bb16186a1c17777bde33e293ae349e0aeffe807e5b22eb65affb7cdd2533"
Jul 30 01:47:36.006435 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:36.006397    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0210bb16186a1c17777bde33e293ae349e0aeffe807e5b22eb65affb7cdd2533"} err="failed to get container status \"0210bb16186a1c17777bde33e293ae349e0aeffe807e5b22eb65affb7cdd2533\": rpc error: code = NotFound desc = could not find container \"0210bb16186a1c17777bde33e293ae349e0aeffe807e5b22eb65affb7cdd2533\": container with ID starting with 0210bb16186a1c17777bde33e293ae349e0aeffe807e5b22eb65affb7cdd2533 not found: ID does not exist"
Jul 30 01:47:36.019168 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:36.019138    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b"]
Jul 30 01:47:36.030502 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:36.026360    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-b64c39-f4f5c5674-jz96b"]
Jul 30 01:47:37.359026 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:37.358992    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="534fb707-c22c-41d9-90bb-ad4dd671e93c" path="/var/lib/kubelet/pods/534fb707-c22c-41d9-90bb-ad4dd671e93c/volumes"
Jul 30 01:47:52.444286 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.444249    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc"]
Jul 30 01:47:52.444978 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.444537    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc" podUID="a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2" containerName="maas-api" containerID="cri-o://ee796a26445648d0d81ecc34cefa03fbf0b9e7ba501508d4e2878cc44410abd0" gracePeriod=30
Jul 30 01:47:52.500329 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.500278    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr"]
Jul 30 01:47:52.500602 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.500550    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr" podUID="d0e44a64-1340-44fc-a03f-cf80443d8854" containerName="payload-processing" containerID="cri-o://890916ab1bf669685f8a35f173a3681033d8afa9bcc3a78c1abe0471366db87c" gracePeriod=30
Jul 30 01:47:52.511637 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.511563    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk"]
Jul 30 01:47:52.511896 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.511785    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk" podUID="f8c06890-b1c5-4062-a793-5ad1ade8739b" containerName="payload-pre-processing" containerID="cri-o://fe0c1ae95d78981c15f2a4659c90b4d604112212dea40ef4ea4add2917436878" gracePeriod=30
Jul 30 01:47:52.788959 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.788933    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc"
Jul 30 01:47:52.808416 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.808381    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr"
Jul 30 01:47:52.815891 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.815864    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk"
Jul 30 01:47:52.914640 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.914604    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0e44a64-1340-44fc-a03f-cf80443d8854-plugins-config-volume\") pod \"d0e44a64-1340-44fc-a03f-cf80443d8854\" (UID: \"d0e44a64-1340-44fc-a03f-cf80443d8854\") "
Jul 30 01:47:52.914832 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.914677    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8c06890-b1c5-4062-a793-5ad1ade8739b-plugins-config-volume\") pod \"f8c06890-b1c5-4062-a793-5ad1ade8739b\" (UID: \"f8c06890-b1c5-4062-a793-5ad1ade8739b\") "
Jul 30 01:47:52.914832 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.914714    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwr6l\" (UniqueName: \"kubernetes.io/projected/a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2-kube-api-access-wwr6l\") pod \"a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2\" (UID: \"a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2\") "
Jul 30 01:47:52.914832 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.914773    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvvv9\" (UniqueName: \"kubernetes.io/projected/d0e44a64-1340-44fc-a03f-cf80443d8854-kube-api-access-vvvv9\") pod \"d0e44a64-1340-44fc-a03f-cf80443d8854\" (UID: \"d0e44a64-1340-44fc-a03f-cf80443d8854\") "
Jul 30 01:47:52.914832 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.914822    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nd6z4\" (UniqueName: \"kubernetes.io/projected/f8c06890-b1c5-4062-a793-5ad1ade8739b-kube-api-access-nd6z4\") pod \"f8c06890-b1c5-4062-a793-5ad1ade8739b\" (UID: \"f8c06890-b1c5-4062-a793-5ad1ade8739b\") "
Jul 30 01:47:52.915067 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.915039    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0e44a64-1340-44fc-a03f-cf80443d8854-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d0e44a64-1340-44fc-a03f-cf80443d8854" (UID: "d0e44a64-1340-44fc-a03f-cf80443d8854"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:47:52.915127 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.915051    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2-maas-api-tls\") pod \"a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2\" (UID: \"a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2\") "
Jul 30 01:47:52.915127 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.915110    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8c06890-b1c5-4062-a793-5ad1ade8739b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f8c06890-b1c5-4062-a793-5ad1ade8739b" (UID: "f8c06890-b1c5-4062-a793-5ad1ade8739b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:47:52.915238 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.915134    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2-openshift-service-ca\") pod \"a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2\" (UID: \"a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2\") "
Jul 30 01:47:52.915471 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.915447    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0e44a64-1340-44fc-a03f-cf80443d8854-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:47:52.915647 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.915473    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8c06890-b1c5-4062-a793-5ad1ade8739b-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:47:52.915647 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.915589    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2" (UID: "a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:47:52.917015 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.916990    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2-kube-api-access-wwr6l" (OuterVolumeSpecName: "kube-api-access-wwr6l") pod "a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2" (UID: "a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2"). InnerVolumeSpecName "kube-api-access-wwr6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:47:52.917152 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.917130    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0e44a64-1340-44fc-a03f-cf80443d8854-kube-api-access-vvvv9" (OuterVolumeSpecName: "kube-api-access-vvvv9") pod "d0e44a64-1340-44fc-a03f-cf80443d8854" (UID: "d0e44a64-1340-44fc-a03f-cf80443d8854"). InnerVolumeSpecName "kube-api-access-vvvv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:47:52.917197 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.917165    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8c06890-b1c5-4062-a793-5ad1ade8739b-kube-api-access-nd6z4" (OuterVolumeSpecName: "kube-api-access-nd6z4") pod "f8c06890-b1c5-4062-a793-5ad1ade8739b" (UID: "f8c06890-b1c5-4062-a793-5ad1ade8739b"). InnerVolumeSpecName "kube-api-access-nd6z4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:47:52.917388 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:52.917370    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2" (UID: "a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:47:53.016799 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.016769    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvvv9\" (UniqueName: \"kubernetes.io/projected/d0e44a64-1340-44fc-a03f-cf80443d8854-kube-api-access-vvvv9\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:47:53.016799 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.016794    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nd6z4\" (UniqueName: \"kubernetes.io/projected/f8c06890-b1c5-4062-a793-5ad1ade8739b-kube-api-access-nd6z4\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:47:53.017078 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.016805    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2-maas-api-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:47:53.017078 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.016842    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2-openshift-service-ca\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:47:53.017078 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.016853    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwr6l\" (UniqueName: \"kubernetes.io/projected/a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2-kube-api-access-wwr6l\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:47:53.094459 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.094421    2546 generic.go:356] "Generic (PLEG): container finished" podID="f8c06890-b1c5-4062-a793-5ad1ade8739b" containerID="fe0c1ae95d78981c15f2a4659c90b4d604112212dea40ef4ea4add2917436878" exitCode=0
Jul 30 01:47:53.094666 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.094471    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk"
Jul 30 01:47:53.094666 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.094510    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk" event={"ID":"f8c06890-b1c5-4062-a793-5ad1ade8739b","Type":"ContainerDied","Data":"fe0c1ae95d78981c15f2a4659c90b4d604112212dea40ef4ea4add2917436878"}
Jul 30 01:47:53.094666 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.094597    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk" event={"ID":"f8c06890-b1c5-4062-a793-5ad1ade8739b","Type":"ContainerDied","Data":"5cc6a2b1ef49ff946aecf60d9ab5d171391fc514ead221047b07e0b1a10590a3"}
Jul 30 01:47:53.094666 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.094626    2546 scope.go:117] "RemoveContainer" containerID="fe0c1ae95d78981c15f2a4659c90b4d604112212dea40ef4ea4add2917436878"
Jul 30 01:47:53.095923 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.095894    2546 generic.go:356] "Generic (PLEG): container finished" podID="a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2" containerID="ee796a26445648d0d81ecc34cefa03fbf0b9e7ba501508d4e2878cc44410abd0" exitCode=0
Jul 30 01:47:53.096062 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.095932    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc" event={"ID":"a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2","Type":"ContainerDied","Data":"ee796a26445648d0d81ecc34cefa03fbf0b9e7ba501508d4e2878cc44410abd0"}
Jul 30 01:47:53.096062 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.095950    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc"
Jul 30 01:47:53.096062 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.095978    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc" event={"ID":"a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2","Type":"ContainerDied","Data":"9d70eab84c63f2ad20c1845221ac9884f90614286409674a5602dcb893cb056e"}
Jul 30 01:47:53.097704 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.097676    2546 generic.go:356] "Generic (PLEG): container finished" podID="d0e44a64-1340-44fc-a03f-cf80443d8854" containerID="890916ab1bf669685f8a35f173a3681033d8afa9bcc3a78c1abe0471366db87c" exitCode=0
Jul 30 01:47:53.097814 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.097729    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr"
Jul 30 01:47:53.097814 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.097798    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr" event={"ID":"d0e44a64-1340-44fc-a03f-cf80443d8854","Type":"ContainerDied","Data":"890916ab1bf669685f8a35f173a3681033d8afa9bcc3a78c1abe0471366db87c"}
Jul 30 01:47:53.097950 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.097839    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr" event={"ID":"d0e44a64-1340-44fc-a03f-cf80443d8854","Type":"ContainerDied","Data":"1df6f810416b671b13f7cfb3373a691aa0a293b695fe90c9ba37450140949fd4"}
Jul 30 01:47:53.105882 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.105861    2546 scope.go:117] "RemoveContainer" containerID="fe0c1ae95d78981c15f2a4659c90b4d604112212dea40ef4ea4add2917436878"
Jul 30 01:47:53.106152 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:47:53.106133    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe0c1ae95d78981c15f2a4659c90b4d604112212dea40ef4ea4add2917436878\": container with ID starting with fe0c1ae95d78981c15f2a4659c90b4d604112212dea40ef4ea4add2917436878 not found: ID does not exist" containerID="fe0c1ae95d78981c15f2a4659c90b4d604112212dea40ef4ea4add2917436878"
Jul 30 01:47:53.106209 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.106160    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe0c1ae95d78981c15f2a4659c90b4d604112212dea40ef4ea4add2917436878"} err="failed to get container status \"fe0c1ae95d78981c15f2a4659c90b4d604112212dea40ef4ea4add2917436878\": rpc error: code = NotFound desc = could not find container \"fe0c1ae95d78981c15f2a4659c90b4d604112212dea40ef4ea4add2917436878\": container with ID starting with fe0c1ae95d78981c15f2a4659c90b4d604112212dea40ef4ea4add2917436878 not found: ID does not exist"
Jul 30 01:47:53.106209 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.106184    2546 scope.go:117] "RemoveContainer" containerID="ee796a26445648d0d81ecc34cefa03fbf0b9e7ba501508d4e2878cc44410abd0"
Jul 30 01:47:53.114727 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.114711    2546 scope.go:117] "RemoveContainer" containerID="ee796a26445648d0d81ecc34cefa03fbf0b9e7ba501508d4e2878cc44410abd0"
Jul 30 01:47:53.114949 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:47:53.114931    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee796a26445648d0d81ecc34cefa03fbf0b9e7ba501508d4e2878cc44410abd0\": container with ID starting with ee796a26445648d0d81ecc34cefa03fbf0b9e7ba501508d4e2878cc44410abd0 not found: ID does not exist" containerID="ee796a26445648d0d81ecc34cefa03fbf0b9e7ba501508d4e2878cc44410abd0"
Jul 30 01:47:53.115023 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.114960    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee796a26445648d0d81ecc34cefa03fbf0b9e7ba501508d4e2878cc44410abd0"} err="failed to get container status \"ee796a26445648d0d81ecc34cefa03fbf0b9e7ba501508d4e2878cc44410abd0\": rpc error: code = NotFound desc = could not find container \"ee796a26445648d0d81ecc34cefa03fbf0b9e7ba501508d4e2878cc44410abd0\": container with ID starting with ee796a26445648d0d81ecc34cefa03fbf0b9e7ba501508d4e2878cc44410abd0 not found: ID does not exist"
Jul 30 01:47:53.115023 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.114981    2546 scope.go:117] "RemoveContainer" containerID="890916ab1bf669685f8a35f173a3681033d8afa9bcc3a78c1abe0471366db87c"
Jul 30 01:47:53.120716 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.120696    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk"]
Jul 30 01:47:53.122874 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.122852    2546 scope.go:117] "RemoveContainer" containerID="890916ab1bf669685f8a35f173a3681033d8afa9bcc3a78c1abe0471366db87c"
Jul 30 01:47:53.123174 ip-10-0-141-85 kubenswrapper[2546]: E0730 01:47:53.123151    2546 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"890916ab1bf669685f8a35f173a3681033d8afa9bcc3a78c1abe0471366db87c\": container with ID starting with 890916ab1bf669685f8a35f173a3681033d8afa9bcc3a78c1abe0471366db87c not found: ID does not exist" containerID="890916ab1bf669685f8a35f173a3681033d8afa9bcc3a78c1abe0471366db87c"
Jul 30 01:47:53.123241 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.123183    2546 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"890916ab1bf669685f8a35f173a3681033d8afa9bcc3a78c1abe0471366db87c"} err="failed to get container status \"890916ab1bf669685f8a35f173a3681033d8afa9bcc3a78c1abe0471366db87c\": rpc error: code = NotFound desc = could not find container \"890916ab1bf669685f8a35f173a3681033d8afa9bcc3a78c1abe0471366db87c\": container with ID starting with 890916ab1bf669685f8a35f173a3681033d8afa9bcc3a78c1abe0471366db87c not found: ID does not exist"
Jul 30 01:47:53.124326 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.124310    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-55b321-c7fb48d76-qgdgk"]
Jul 30 01:47:53.133932 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.133910    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr"]
Jul 30 01:47:53.135786 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.135769    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-55b321-7ff7b769f4-dkgqr"]
Jul 30 01:47:53.145588 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.145570    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc"]
Jul 30 01:47:53.149122 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.149102    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-55b321-dc778f7d5-5gnkc"]
Jul 30 01:47:53.363178 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.363101    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2" path="/var/lib/kubelet/pods/a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2/volumes"
Jul 30 01:47:53.363490 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.363475    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0e44a64-1340-44fc-a03f-cf80443d8854" path="/var/lib/kubelet/pods/d0e44a64-1340-44fc-a03f-cf80443d8854/volumes"
Jul 30 01:47:53.363810 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:47:53.363798    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8c06890-b1c5-4062-a793-5ad1ade8739b" path="/var/lib/kubelet/pods/f8c06890-b1c5-4062-a793-5ad1ade8739b/volumes"
Jul 30 01:48:09.951577 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:09.951540    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f"]
Jul 30 01:48:09.952167 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:09.951829    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f" podUID="44825109-2ad0-4401-845e-0bd91f61256f" containerName="maas-api" containerID="cri-o://6074fb1a8e4ef366229d8d440bd96a62aa2fbd19c665fd73cb5760231e7c3cdf" gracePeriod=30
Jul 30 01:48:10.007492 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.007449    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn"]
Jul 30 01:48:10.007720 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.007689    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn" podUID="f0ceb8f3-69e9-4581-8312-57701cea8163" containerName="payload-processing" containerID="cri-o://598925a0ed14bc3fb1bfe960b7b6ccb3ce27425d86237025a96d3ea3d4a48476" gracePeriod=30
Jul 30 01:48:10.015846 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.015816    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd"]
Jul 30 01:48:10.016153 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.016101    2546 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd" podUID="dd93670e-7cc5-4c60-be8b-606e79fa5eb0" containerName="payload-pre-processing" containerID="cri-o://8a22b288267b148ffe0dea0273e7e8a598c46108274f0e02970a1dfaad72ca29" gracePeriod=30
Jul 30 01:48:10.194281 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.194241    2546 generic.go:356] "Generic (PLEG): container finished" podID="dd93670e-7cc5-4c60-be8b-606e79fa5eb0" containerID="8a22b288267b148ffe0dea0273e7e8a598c46108274f0e02970a1dfaad72ca29" exitCode=0
Jul 30 01:48:10.194439 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.194358    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd" event={"ID":"dd93670e-7cc5-4c60-be8b-606e79fa5eb0","Type":"ContainerDied","Data":"8a22b288267b148ffe0dea0273e7e8a598c46108274f0e02970a1dfaad72ca29"}
Jul 30 01:48:10.196102 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.196066    2546 generic.go:356] "Generic (PLEG): container finished" podID="44825109-2ad0-4401-845e-0bd91f61256f" containerID="6074fb1a8e4ef366229d8d440bd96a62aa2fbd19c665fd73cb5760231e7c3cdf" exitCode=0
Jul 30 01:48:10.196242 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.196154    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f" event={"ID":"44825109-2ad0-4401-845e-0bd91f61256f","Type":"ContainerDied","Data":"6074fb1a8e4ef366229d8d440bd96a62aa2fbd19c665fd73cb5760231e7c3cdf"}
Jul 30 01:48:10.197748 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.197723    2546 generic.go:356] "Generic (PLEG): container finished" podID="f0ceb8f3-69e9-4581-8312-57701cea8163" containerID="598925a0ed14bc3fb1bfe960b7b6ccb3ce27425d86237025a96d3ea3d4a48476" exitCode=0
Jul 30 01:48:10.197852 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.197807    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn" event={"ID":"f0ceb8f3-69e9-4581-8312-57701cea8163","Type":"ContainerDied","Data":"598925a0ed14bc3fb1bfe960b7b6ccb3ce27425d86237025a96d3ea3d4a48476"}
Jul 30 01:48:10.304290 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.304262    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn"
Jul 30 01:48:10.309643 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.309624    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f"
Jul 30 01:48:10.321556 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.321507    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd"
Jul 30 01:48:10.468011 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.467984    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd93670e-7cc5-4c60-be8b-606e79fa5eb0-plugins-config-volume\") pod \"dd93670e-7cc5-4c60-be8b-606e79fa5eb0\" (UID: \"dd93670e-7cc5-4c60-be8b-606e79fa5eb0\") "
Jul 30 01:48:10.468188 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.468067    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44825109-2ad0-4401-845e-0bd91f61256f-maas-api-tls\") pod \"44825109-2ad0-4401-845e-0bd91f61256f\" (UID: \"44825109-2ad0-4401-845e-0bd91f61256f\") "
Jul 30 01:48:10.468188 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.468092    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcn9r\" (UniqueName: \"kubernetes.io/projected/dd93670e-7cc5-4c60-be8b-606e79fa5eb0-kube-api-access-fcn9r\") pod \"dd93670e-7cc5-4c60-be8b-606e79fa5eb0\" (UID: \"dd93670e-7cc5-4c60-be8b-606e79fa5eb0\") "
Jul 30 01:48:10.468188 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.468110    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjczg\" (UniqueName: \"kubernetes.io/projected/f0ceb8f3-69e9-4581-8312-57701cea8163-kube-api-access-qjczg\") pod \"f0ceb8f3-69e9-4581-8312-57701cea8163\" (UID: \"f0ceb8f3-69e9-4581-8312-57701cea8163\") "
Jul 30 01:48:10.468188 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.468159    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44825109-2ad0-4401-845e-0bd91f61256f-openshift-service-ca\") pod \"44825109-2ad0-4401-845e-0bd91f61256f\" (UID: \"44825109-2ad0-4401-845e-0bd91f61256f\") "
Jul 30 01:48:10.468188 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.468179    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0ceb8f3-69e9-4581-8312-57701cea8163-plugins-config-volume\") pod \"f0ceb8f3-69e9-4581-8312-57701cea8163\" (UID: \"f0ceb8f3-69e9-4581-8312-57701cea8163\") "
Jul 30 01:48:10.468566 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.468220    2546 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-284sb\" (UniqueName: \"kubernetes.io/projected/44825109-2ad0-4401-845e-0bd91f61256f-kube-api-access-284sb\") pod \"44825109-2ad0-4401-845e-0bd91f61256f\" (UID: \"44825109-2ad0-4401-845e-0bd91f61256f\") "
Jul 30 01:48:10.468566 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.468427    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd93670e-7cc5-4c60-be8b-606e79fa5eb0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dd93670e-7cc5-4c60-be8b-606e79fa5eb0" (UID: "dd93670e-7cc5-4c60-be8b-606e79fa5eb0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:48:10.468700 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.468583    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44825109-2ad0-4401-845e-0bd91f61256f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "44825109-2ad0-4401-845e-0bd91f61256f" (UID: "44825109-2ad0-4401-845e-0bd91f61256f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:48:10.468700 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.468672    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0ceb8f3-69e9-4581-8312-57701cea8163-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f0ceb8f3-69e9-4581-8312-57701cea8163" (UID: "f0ceb8f3-69e9-4581-8312-57701cea8163"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 01:48:10.470253 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.470220    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0ceb8f3-69e9-4581-8312-57701cea8163-kube-api-access-qjczg" (OuterVolumeSpecName: "kube-api-access-qjczg") pod "f0ceb8f3-69e9-4581-8312-57701cea8163" (UID: "f0ceb8f3-69e9-4581-8312-57701cea8163"). InnerVolumeSpecName "kube-api-access-qjczg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:48:10.470357 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.470269    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44825109-2ad0-4401-845e-0bd91f61256f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "44825109-2ad0-4401-845e-0bd91f61256f" (UID: "44825109-2ad0-4401-845e-0bd91f61256f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 01:48:10.470357 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.470280    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd93670e-7cc5-4c60-be8b-606e79fa5eb0-kube-api-access-fcn9r" (OuterVolumeSpecName: "kube-api-access-fcn9r") pod "dd93670e-7cc5-4c60-be8b-606e79fa5eb0" (UID: "dd93670e-7cc5-4c60-be8b-606e79fa5eb0"). InnerVolumeSpecName "kube-api-access-fcn9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:48:10.470357 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.470306    2546 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44825109-2ad0-4401-845e-0bd91f61256f-kube-api-access-284sb" (OuterVolumeSpecName: "kube-api-access-284sb") pod "44825109-2ad0-4401-845e-0bd91f61256f" (UID: "44825109-2ad0-4401-845e-0bd91f61256f"). InnerVolumeSpecName "kube-api-access-284sb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 01:48:10.569013 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.568942    2546 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44825109-2ad0-4401-845e-0bd91f61256f-openshift-service-ca\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:48:10.569013 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.568967    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0ceb8f3-69e9-4581-8312-57701cea8163-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:48:10.569013 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.568979    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-284sb\" (UniqueName: \"kubernetes.io/projected/44825109-2ad0-4401-845e-0bd91f61256f-kube-api-access-284sb\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:48:10.569013 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.568988    2546 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd93670e-7cc5-4c60-be8b-606e79fa5eb0-plugins-config-volume\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:48:10.569013 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.568997    2546 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44825109-2ad0-4401-845e-0bd91f61256f-maas-api-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:48:10.569013 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.569007    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcn9r\" (UniqueName: \"kubernetes.io/projected/dd93670e-7cc5-4c60-be8b-606e79fa5eb0-kube-api-access-fcn9r\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:48:10.569013 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:10.569017    2546 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjczg\" (UniqueName: \"kubernetes.io/projected/f0ceb8f3-69e9-4581-8312-57701cea8163-kube-api-access-qjczg\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\""
Jul 30 01:48:11.204377 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:11.204348    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd"
Jul 30 01:48:11.204890 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:11.204347    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd" event={"ID":"dd93670e-7cc5-4c60-be8b-606e79fa5eb0","Type":"ContainerDied","Data":"5ac2f6d9d48b5182dd8382e7e577721c718a7951798bbff383223ea98a244e7e"}
Jul 30 01:48:11.204890 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:11.204466    2546 scope.go:117] "RemoveContainer" containerID="8a22b288267b148ffe0dea0273e7e8a598c46108274f0e02970a1dfaad72ca29"
Jul 30 01:48:11.205963 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:11.205790    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f"
Jul 30 01:48:11.205963 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:11.205805    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f" event={"ID":"44825109-2ad0-4401-845e-0bd91f61256f","Type":"ContainerDied","Data":"b5dfee95316ede9ec47e8bcfaa60c3375b1243794eef77aa62a206f135c57967"}
Jul 30 01:48:11.207215 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:11.207186    2546 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn"
Jul 30 01:48:11.207215 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:11.207199    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn" event={"ID":"f0ceb8f3-69e9-4581-8312-57701cea8163","Type":"ContainerDied","Data":"06d05b869ff57297b2884bbdb37e8b884c6da8d92e6e17989d4b89970dc38770"}
Jul 30 01:48:11.216115 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:11.215112    2546 scope.go:117] "RemoveContainer" containerID="6074fb1a8e4ef366229d8d440bd96a62aa2fbd19c665fd73cb5760231e7c3cdf"
Jul 30 01:48:11.228119 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:11.228098    2546 scope.go:117] "RemoveContainer" containerID="598925a0ed14bc3fb1bfe960b7b6ccb3ce27425d86237025a96d3ea3d4a48476"
Jul 30 01:48:11.236982 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:11.236965    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f"]
Jul 30 01:48:11.241130 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:11.241109    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-fba93c-76f6bb5966-b9k9f"]
Jul 30 01:48:11.250241 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:11.250221    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd"]
Jul 30 01:48:11.254364 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:11.254343    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-fba93c-5c946cf868-lppnd"]
Jul 30 01:48:11.264797 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:11.264778    2546 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn"]
Jul 30 01:48:11.270579 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:11.270560    2546 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-fba93c-5d587bf898-pslsn"]
Jul 30 01:48:11.358130 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:11.358099    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44825109-2ad0-4401-845e-0bd91f61256f" path="/var/lib/kubelet/pods/44825109-2ad0-4401-845e-0bd91f61256f/volumes"
Jul 30 01:48:11.358466 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:11.358453    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd93670e-7cc5-4c60-be8b-606e79fa5eb0" path="/var/lib/kubelet/pods/dd93670e-7cc5-4c60-be8b-606e79fa5eb0/volumes"
Jul 30 01:48:11.358802 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:11.358789    2546 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0ceb8f3-69e9-4581-8312-57701cea8163" path="/var/lib/kubelet/pods/f0ceb8f3-69e9-4581-8312-57701cea8163/volumes"
Jul 30 01:48:22.483257 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:22.483226    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-7c4q4_ea6158a9-c3f8-4cb1-a9c6-6425a2bf5bcf/manager/0.log"
Jul 30 01:48:22.602810 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:22.602710    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7pmdr_0c69ed81-87c8-4800-8640-6e54b1267eeb/manager/0.log"
Jul 30 01:48:22.741832 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:22.741751    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-756cdf68f8-smlzn_2bcc1a1a-24a5-4fa3-8d47-c2a498047a18/manager/0.log"
Jul 30 01:48:22.891928 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:22.891898    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-hbwcm_2469cd23-8223-4779-a4ac-4268d35322db/server/0.log"
Jul 30 01:48:23.018695 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:23.018575    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-jt7jk_7902a946-c204-49e0-a986-fd5b0db8011c/manager/0.log"
Jul 30 01:48:23.129651 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:23.129623    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-746b4b6684-9phwj_bdf82d5b-7f06-4b71-862d-5a9aeac97c71/copy-manifests/0.log"
Jul 30 01:48:23.148110 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:23.148058    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-746b4b6684-9phwj_bdf82d5b-7f06-4b71-862d-5a9aeac97c71/manager/0.log"
Jul 30 01:48:23.723496 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:23.723465    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6d87d4f98c-gccf6_987f637e-b9af-48d1-b449-f0e95f618cb4/maas-api/0.log"
Jul 30 01:48:24.156635 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:24.156556    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-x2brv_fc0b6562-aad5-400e-bb56-fba4d2516b38/postgres/0.log"
Jul 30 01:48:25.537267 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:25.537232    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-7mhtx_3e06937f-2029-46d1-90aa-17befcb538e1/manager/0.log"
Jul 30 01:48:25.648664 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:25.648637    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-nx2jl_057120b0-8df7-4900-ad80-9d9bad5afbd1/manager/0.log"
Jul 30 01:48:25.877643 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:25.877566    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-68bvq_f3851584-c3a7-48ec-b3ab-f43becc55337/registry-server/0.log"
Jul 30 01:48:26.004696 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:26.004668    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-bfwrv_147f1e75-25ab-45d1-8e8c-475c4327194f/manager/0.log"
Jul 30 01:48:26.239808 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:26.239778    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-x5dr5_392882c7-8ab1-49d1-8de2-510b564fee31/manager/0.log"
Jul 30 01:48:26.908906 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:26.908878    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-95755dfb5-nf8t8_8563107f-b595-483e-aa72-9a6be121a415/kube-auth-proxy/0.log"
Jul 30 01:48:27.149974 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:27.149937    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-sfjl8_fa8377ea-dafa-4afb-adfc-49a6ee20d360/payload-pre-processing/0.log"
Jul 30 01:48:27.269881 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:27.269847    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-5692l_4c08d630-9e0a-412c-8531-968e388ff89e/payload-processing/0.log"
Jul 30 01:48:27.724678 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:27.724643    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-x4tw9_026a16bb-b4ca-44b3-99b1-9ba46f2f9c6b/main/0.log"
Jul 30 01:48:28.051223 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:28.051139    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fplspt_b7cf65d6-77fa-4736-8b8e-b6fb863ed4d7/main/0.log"
Jul 30 01:48:33.442339 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442307    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kwqhr/must-gather-cv2gk"]
Jul 30 01:48:33.442738 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442704    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0e44a64-1340-44fc-a03f-cf80443d8854" containerName="payload-processing"
Jul 30 01:48:33.442738 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442718    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0e44a64-1340-44fc-a03f-cf80443d8854" containerName="payload-processing"
Jul 30 01:48:33.442738 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442730    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51b2447f-118b-4b88-af9e-4b1e36c7293e" containerName="payload-pre-processing"
Jul 30 01:48:33.442738 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442736    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="51b2447f-118b-4b88-af9e-4b1e36c7293e" containerName="payload-pre-processing"
Jul 30 01:48:33.442876 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442744    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ce4a97c-37f4-43f4-aecf-507f685c67b4" containerName="maas-api"
Jul 30 01:48:33.442876 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442750    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ce4a97c-37f4-43f4-aecf-507f685c67b4" containerName="maas-api"
Jul 30 01:48:33.442876 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442757    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44825109-2ad0-4401-845e-0bd91f61256f" containerName="maas-api"
Jul 30 01:48:33.442876 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442762    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="44825109-2ad0-4401-845e-0bd91f61256f" containerName="maas-api"
Jul 30 01:48:33.442876 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442768    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0ceb8f3-69e9-4581-8312-57701cea8163" containerName="payload-processing"
Jul 30 01:48:33.442876 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442774    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0ceb8f3-69e9-4581-8312-57701cea8163" containerName="payload-processing"
Jul 30 01:48:33.442876 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442782    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd93670e-7cc5-4c60-be8b-606e79fa5eb0" containerName="payload-pre-processing"
Jul 30 01:48:33.442876 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442789    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd93670e-7cc5-4c60-be8b-606e79fa5eb0" containerName="payload-pre-processing"
Jul 30 01:48:33.442876 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442798    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2" containerName="maas-api"
Jul 30 01:48:33.442876 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442805    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2" containerName="maas-api"
Jul 30 01:48:33.442876 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442814    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2eb2c373-ba22-49fa-9968-3184a6272bf9" containerName="payload-processing"
Jul 30 01:48:33.442876 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442819    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="2eb2c373-ba22-49fa-9968-3184a6272bf9" containerName="payload-processing"
Jul 30 01:48:33.442876 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442831    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8c06890-b1c5-4062-a793-5ad1ade8739b" containerName="payload-pre-processing"
Jul 30 01:48:33.442876 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442836    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8c06890-b1c5-4062-a793-5ad1ade8739b" containerName="payload-pre-processing"
Jul 30 01:48:33.442876 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442843    2546 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="534fb707-c22c-41d9-90bb-ad4dd671e93c" containerName="payload-processing"
Jul 30 01:48:33.442876 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442849    2546 state_mem.go:107] "Deleted CPUSet assignment" podUID="534fb707-c22c-41d9-90bb-ad4dd671e93c" containerName="payload-processing"
Jul 30 01:48:33.443358 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442910    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="a44f902c-4e2e-44d2-a3ef-87e8c70cc7c2" containerName="maas-api"
Jul 30 01:48:33.443358 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442918    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="2eb2c373-ba22-49fa-9968-3184a6272bf9" containerName="payload-processing"
Jul 30 01:48:33.443358 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442925    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ce4a97c-37f4-43f4-aecf-507f685c67b4" containerName="maas-api"
Jul 30 01:48:33.443358 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442931    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="51b2447f-118b-4b88-af9e-4b1e36c7293e" containerName="payload-pre-processing"
Jul 30 01:48:33.443358 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442938    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="534fb707-c22c-41d9-90bb-ad4dd671e93c" containerName="payload-processing"
Jul 30 01:48:33.443358 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442943    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8c06890-b1c5-4062-a793-5ad1ade8739b" containerName="payload-pre-processing"
Jul 30 01:48:33.443358 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442948    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="44825109-2ad0-4401-845e-0bd91f61256f" containerName="maas-api"
Jul 30 01:48:33.443358 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442954    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd93670e-7cc5-4c60-be8b-606e79fa5eb0" containerName="payload-pre-processing"
Jul 30 01:48:33.443358 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442959    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0ceb8f3-69e9-4581-8312-57701cea8163" containerName="payload-processing"
Jul 30 01:48:33.443358 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.442965    2546 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0e44a64-1340-44fc-a03f-cf80443d8854" containerName="payload-processing"
Jul 30 01:48:33.446180 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.446164    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kwqhr/must-gather-cv2gk"
Jul 30 01:48:33.448833 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.448807    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kwqhr\"/\"kube-root-ca.crt\""
Jul 30 01:48:33.448833 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.448840    2546 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kwqhr\"/\"openshift-service-ca.crt\""
Jul 30 01:48:33.449023 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.448808    2546 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kwqhr\"/\"default-dockercfg-hsst4\""
Jul 30 01:48:33.455864 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.455840    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kwqhr/must-gather-cv2gk"]
Jul 30 01:48:33.549557 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.549530    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 01:48:33.557572 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.557549    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 01:48:33.570694 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.570674    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f2e71a52-c459-49a4-9db9-61d69615f692-must-gather-output\") pod \"must-gather-cv2gk\" (UID: \"f2e71a52-c459-49a4-9db9-61d69615f692\") " pod="openshift-must-gather-kwqhr/must-gather-cv2gk"
Jul 30 01:48:33.570811 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.570724    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kbml\" (UniqueName: \"kubernetes.io/projected/f2e71a52-c459-49a4-9db9-61d69615f692-kube-api-access-9kbml\") pod \"must-gather-cv2gk\" (UID: \"f2e71a52-c459-49a4-9db9-61d69615f692\") " pod="openshift-must-gather-kwqhr/must-gather-cv2gk"
Jul 30 01:48:33.671333 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.671300    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kbml\" (UniqueName: \"kubernetes.io/projected/f2e71a52-c459-49a4-9db9-61d69615f692-kube-api-access-9kbml\") pod \"must-gather-cv2gk\" (UID: \"f2e71a52-c459-49a4-9db9-61d69615f692\") " pod="openshift-must-gather-kwqhr/must-gather-cv2gk"
Jul 30 01:48:33.671479 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.671367    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f2e71a52-c459-49a4-9db9-61d69615f692-must-gather-output\") pod \"must-gather-cv2gk\" (UID: \"f2e71a52-c459-49a4-9db9-61d69615f692\") " pod="openshift-must-gather-kwqhr/must-gather-cv2gk"
Jul 30 01:48:33.671688 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.671672    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f2e71a52-c459-49a4-9db9-61d69615f692-must-gather-output\") pod \"must-gather-cv2gk\" (UID: \"f2e71a52-c459-49a4-9db9-61d69615f692\") " pod="openshift-must-gather-kwqhr/must-gather-cv2gk"
Jul 30 01:48:33.679596 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.679569    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kbml\" (UniqueName: \"kubernetes.io/projected/f2e71a52-c459-49a4-9db9-61d69615f692-kube-api-access-9kbml\") pod \"must-gather-cv2gk\" (UID: \"f2e71a52-c459-49a4-9db9-61d69615f692\") " pod="openshift-must-gather-kwqhr/must-gather-cv2gk"
Jul 30 01:48:33.755659 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.755635    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kwqhr/must-gather-cv2gk"
Jul 30 01:48:33.894071 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.894049    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kwqhr/must-gather-cv2gk"]
Jul 30 01:48:33.896302 ip-10-0-141-85 kubenswrapper[2546]: W0730 01:48:33.896279    2546 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2e71a52_c459_49a4_9db9_61d69615f692.slice/crio-3fa5d0da928ddc933238e2e178510bcee29e3a059c45f287e47e7300d59e4f55 WatchSource:0}: Error finding container 3fa5d0da928ddc933238e2e178510bcee29e3a059c45f287e47e7300d59e4f55: Status 404 returned error can't find the container with id 3fa5d0da928ddc933238e2e178510bcee29e3a059c45f287e47e7300d59e4f55
Jul 30 01:48:33.898375 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:33.898358    2546 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 01:48:34.327751 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:34.327717    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kwqhr/must-gather-cv2gk" event={"ID":"f2e71a52-c459-49a4-9db9-61d69615f692","Type":"ContainerStarted","Data":"3fa5d0da928ddc933238e2e178510bcee29e3a059c45f287e47e7300d59e4f55"}
Jul 30 01:48:35.344961 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:35.344359    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kwqhr/must-gather-cv2gk" event={"ID":"f2e71a52-c459-49a4-9db9-61d69615f692","Type":"ContainerStarted","Data":"e30f4b8e8e167ff729cd0def0862b9ab60f8bdaf780598c3e12be4ecce68dcd5"}
Jul 30 01:48:35.344961 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:35.344417    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kwqhr/must-gather-cv2gk" event={"ID":"f2e71a52-c459-49a4-9db9-61d69615f692","Type":"ContainerStarted","Data":"2d839d990992bab7e114dc38b1a7597b1207508b7bee5b8105c8bd331f99cfc1"}
Jul 30 01:48:35.361960 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:35.361885    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kwqhr/must-gather-cv2gk" podStartSLOduration=1.507305549 podStartE2EDuration="2.361865255s" totalImagesPullingTime="854.559706ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 01:48:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:48:35.361163085 +0000 UTC m=+3902.583896904" watchObservedRunningTime="2026-07-30 01:48:35.361865255 +0000 UTC m=+3902.584599075"
Jul 30 01:48:36.287745 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:36.287709    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9lfz5_b7286a2e-d6a2-4db7-912a-3ae863862afa/global-pull-secret-syncer/0.log"
Jul 30 01:48:36.497803 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:36.497766    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vkxdr_7eaeb5a9-feed-476d-8a67-083665883cdf/konnectivity-agent/0.log"
Jul 30 01:48:36.574377 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:36.574285    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-85.ec2.internal_80e212081188b2e37af81e25e05fed3c/haproxy/0.log"
Jul 30 01:48:41.897302 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:41.897263    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-7mhtx_3e06937f-2029-46d1-90aa-17befcb538e1/manager/0.log"
Jul 30 01:48:41.921830 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:41.921798    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-nx2jl_057120b0-8df7-4900-ad80-9d9bad5afbd1/manager/0.log"
Jul 30 01:48:42.025246 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:42.025032    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-68bvq_f3851584-c3a7-48ec-b3ab-f43becc55337/registry-server/0.log"
Jul 30 01:48:42.147249 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:42.147207    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-bfwrv_147f1e75-25ab-45d1-8e8c-475c4327194f/manager/0.log"
Jul 30 01:48:42.293612 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:42.293575    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-x5dr5_392882c7-8ab1-49d1-8de2-510b564fee31/manager/0.log"
Jul 30 01:48:44.329235 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:44.329189    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ktk26_da33b631-3b5b-42e9-8599-7df6bcf8fc9e/node-exporter/0.log"
Jul 30 01:48:44.352216 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:44.352181    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ktk26_da33b631-3b5b-42e9-8599-7df6bcf8fc9e/kube-rbac-proxy/0.log"
Jul 30 01:48:44.371535 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:44.371484    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ktk26_da33b631-3b5b-42e9-8599-7df6bcf8fc9e/init-textfile/0.log"
Jul 30 01:48:46.144595 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:46.144563    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-58x9b_9254a1f8-d810-4bea-837c-2698526cb5d4/networking-console-plugin/0.log"
Jul 30 01:48:46.427488 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:46.427389    2546 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj"]
Jul 30 01:48:46.435951 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:46.435923    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj"
Jul 30 01:48:46.440845 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:46.440813    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj"]
Jul 30 01:48:46.507068 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:46.507028    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tn4c8\" (UniqueName: \"kubernetes.io/projected/bb779820-9909-4191-a143-f71fbc34de80-kube-api-access-tn4c8\") pod \"perf-node-gather-daemonset-6pljj\" (UID: \"bb779820-9909-4191-a143-f71fbc34de80\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj"
Jul 30 01:48:46.507381 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:46.507357    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bb779820-9909-4191-a143-f71fbc34de80-proc\") pod \"perf-node-gather-daemonset-6pljj\" (UID: \"bb779820-9909-4191-a143-f71fbc34de80\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj"
Jul 30 01:48:46.507879 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:46.507860    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb779820-9909-4191-a143-f71fbc34de80-sys\") pod \"perf-node-gather-daemonset-6pljj\" (UID: \"bb779820-9909-4191-a143-f71fbc34de80\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj"
Jul 30 01:48:46.507980 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:46.507926    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb779820-9909-4191-a143-f71fbc34de80-lib-modules\") pod \"perf-node-gather-daemonset-6pljj\" (UID: \"bb779820-9909-4191-a143-f71fbc34de80\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj"
Jul 30 01:48:46.508053 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:46.508003    2546 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bb779820-9909-4191-a143-f71fbc34de80-podres\") pod \"perf-node-gather-daemonset-6pljj\" (UID: \"bb779820-9909-4191-a143-f71fbc34de80\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj"
Jul 30 01:48:46.609088 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:46.609038    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bb779820-9909-4191-a143-f71fbc34de80-podres\") pod \"perf-node-gather-daemonset-6pljj\" (UID: \"bb779820-9909-4191-a143-f71fbc34de80\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj"
Jul 30 01:48:46.609287 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:46.609168    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn4c8\" (UniqueName: \"kubernetes.io/projected/bb779820-9909-4191-a143-f71fbc34de80-kube-api-access-tn4c8\") pod \"perf-node-gather-daemonset-6pljj\" (UID: \"bb779820-9909-4191-a143-f71fbc34de80\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj"
Jul 30 01:48:46.609287 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:46.609218    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bb779820-9909-4191-a143-f71fbc34de80-proc\") pod \"perf-node-gather-daemonset-6pljj\" (UID: \"bb779820-9909-4191-a143-f71fbc34de80\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj"
Jul 30 01:48:46.609287 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:46.609249    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb779820-9909-4191-a143-f71fbc34de80-sys\") pod \"perf-node-gather-daemonset-6pljj\" (UID: \"bb779820-9909-4191-a143-f71fbc34de80\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj"
Jul 30 01:48:46.609287 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:46.609275    2546 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb779820-9909-4191-a143-f71fbc34de80-lib-modules\") pod \"perf-node-gather-daemonset-6pljj\" (UID: \"bb779820-9909-4191-a143-f71fbc34de80\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj"
Jul 30 01:48:46.609561 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:46.609446    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb779820-9909-4191-a143-f71fbc34de80-lib-modules\") pod \"perf-node-gather-daemonset-6pljj\" (UID: \"bb779820-9909-4191-a143-f71fbc34de80\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj"
Jul 30 01:48:46.609632 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:46.609571    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bb779820-9909-4191-a143-f71fbc34de80-podres\") pod \"perf-node-gather-daemonset-6pljj\" (UID: \"bb779820-9909-4191-a143-f71fbc34de80\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj"
Jul 30 01:48:46.609928 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:46.609904    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bb779820-9909-4191-a143-f71fbc34de80-proc\") pod \"perf-node-gather-daemonset-6pljj\" (UID: \"bb779820-9909-4191-a143-f71fbc34de80\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj"
Jul 30 01:48:46.610003 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:46.609969    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb779820-9909-4191-a143-f71fbc34de80-sys\") pod \"perf-node-gather-daemonset-6pljj\" (UID: \"bb779820-9909-4191-a143-f71fbc34de80\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj"
Jul 30 01:48:46.619500 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:46.618083    2546 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tn4c8\" (UniqueName: \"kubernetes.io/projected/bb779820-9909-4191-a143-f71fbc34de80-kube-api-access-tn4c8\") pod \"perf-node-gather-daemonset-6pljj\" (UID: \"bb779820-9909-4191-a143-f71fbc34de80\") " pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj"
Jul 30 01:48:46.757075 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:46.757037    2546 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj"
Jul 30 01:48:46.781601 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:46.781401    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/2.log"
Jul 30 01:48:46.791690 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:46.791654    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wx49h_c56ae285-bf61-4f11-b846-827ea2e7594e/console-operator/3.log"
Jul 30 01:48:46.971847 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:46.971633    2546 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj"]
Jul 30 01:48:47.353297 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:47.353220    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-c85d6c89f-ftlbz_d29bb327-c2f4-4488-8e2e-f96d3bdaef42/console/0.log"
Jul 30 01:48:47.439117 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:47.439068    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj" event={"ID":"bb779820-9909-4191-a143-f71fbc34de80","Type":"ContainerStarted","Data":"0cc66e8d36e88baf1cbe57523ca5be9efb3511db2648869177e9cd4e6e0134d0"}
Jul 30 01:48:47.439337 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:47.439155    2546 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj" event={"ID":"bb779820-9909-4191-a143-f71fbc34de80","Type":"ContainerStarted","Data":"cc396de3452bdc29eb6a35a024595e541d27a2535bf46aa7a7182e73f38502ab"}
Jul 30 01:48:47.439414 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:47.439364    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj"
Jul 30 01:48:47.457351 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:47.457292    2546 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj" podStartSLOduration=1.457271378 podStartE2EDuration="1.457271378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 01:48:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 01:48:47.454302695 +0000 UTC m=+3914.677036515" watchObservedRunningTime="2026-07-30 01:48:47.457271378 +0000 UTC m=+3914.680005197"
Jul 30 01:48:48.923207 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:48.923177    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4xhnp_4d56a868-73af-4d44-946e-13aa14d16532/dns/0.log"
Jul 30 01:48:48.946485 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:48.946458    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4xhnp_4d56a868-73af-4d44-946e-13aa14d16532/kube-rbac-proxy/0.log"
Jul 30 01:48:49.144942 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:49.144909    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-st67h_e0c213ea-4c65-4b0c-813c-0ae9e69047e1/dns-node-resolver/0.log"
Jul 30 01:48:49.788046 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:49.788013    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7vhrw_b429caa1-4e6d-4f0c-b426-1e8b75379b8b/node-ca/0.log"
Jul 30 01:48:50.987151 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:50.987116    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-95755dfb5-nf8t8_8563107f-b595-483e-aa72-9a6be121a415/kube-auth-proxy/0.log"
Jul 30 01:48:51.084347 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:51.084309    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-sfjl8_fa8377ea-dafa-4afb-adfc-49a6ee20d360/payload-pre-processing/0.log"
Jul 30 01:48:51.123236 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:51.123198    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-5692l_4c08d630-9e0a-412c-8531-968e388ff89e/payload-processing/0.log"
Jul 30 01:48:51.930437 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:51.930410    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dptvp_d2b21e5b-4b93-49d1-a49e-51d05d60905f/serve-healthcheck-canary/0.log"
Jul 30 01:48:52.424956 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:52.424905    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-8qbqm_8fd5e744-a7db-428d-9d46-c2413213a511/insights-operator/0.log"
Jul 30 01:48:52.426158 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:52.426126    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-8qbqm_8fd5e744-a7db-428d-9d46-c2413213a511/insights-operator/1.log"
Jul 30 01:48:52.513060 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:52.513029    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jhdjl_42841174-db24-4585-a998-29845763f1e4/kube-rbac-proxy/0.log"
Jul 30 01:48:52.531936 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:52.531905    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jhdjl_42841174-db24-4585-a998-29845763f1e4/exporter/0.log"
Jul 30 01:48:52.552238 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:52.552210    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jhdjl_42841174-db24-4585-a998-29845763f1e4/extractor/0.log"
Jul 30 01:48:53.458438 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:53.458411    2546 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kwqhr/perf-node-gather-daemonset-6pljj"
Jul 30 01:48:54.619550 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:54.619488    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-7c4q4_ea6158a9-c3f8-4cb1-a9c6-6425a2bf5bcf/manager/0.log"
Jul 30 01:48:55.049042 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:55.049006    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7pmdr_0c69ed81-87c8-4800-8640-6e54b1267eeb/manager/0.log"
Jul 30 01:48:55.344642 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:55.344553    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-756cdf68f8-smlzn_2bcc1a1a-24a5-4fa3-8d47-c2a498047a18/manager/0.log"
Jul 30 01:48:55.366916 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:55.366880    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-hbwcm_2469cd23-8223-4779-a4ac-4268d35322db/server/0.log"
Jul 30 01:48:55.491245 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:55.491209    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-jt7jk_7902a946-c204-49e0-a986-fd5b0db8011c/manager/0.log"
Jul 30 01:48:55.661099 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:55.661060    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-746b4b6684-9phwj_bdf82d5b-7f06-4b71-862d-5a9aeac97c71/manager/0.log"
Jul 30 01:48:55.679099 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:55.679038    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-746b4b6684-9phwj_bdf82d5b-7f06-4b71-862d-5a9aeac97c71/copy-manifests/0.log"
Jul 30 01:48:57.203339 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:57.203310    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-xslf4_a92e23ed-7c35-45fc-8121-69918812e3bb/openshift-lws-operator/0.log"
Jul 30 01:48:57.271536 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:57.271451    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-95755dfb5-nf8t8_8563107f-b595-483e-aa72-9a6be121a415/kube-auth-proxy/0.log"
Jul 30 01:48:57.360744 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:57.360710    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-sfjl8_fa8377ea-dafa-4afb-adfc-49a6ee20d360/payload-pre-processing/0.log"
Jul 30 01:48:57.395189 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:57.395162    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-5692l_4c08d630-9e0a-412c-8531-968e388ff89e/payload-processing/0.log"
Jul 30 01:48:58.992148 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:58.992113    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-95755dfb5-nf8t8_8563107f-b595-483e-aa72-9a6be121a415/kube-auth-proxy/0.log"
Jul 30 01:48:59.065650 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:59.065615    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-sfjl8_fa8377ea-dafa-4afb-adfc-49a6ee20d360/payload-pre-processing/0.log"
Jul 30 01:48:59.094831 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:48:59.094799    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-5692l_4c08d630-9e0a-412c-8531-968e388ff89e/payload-processing/0.log"
Jul 30 01:49:02.139963 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:49:02.139904    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-x85f9_24e359fe-816f-4977-a4e6-3d3bb82a1634/migrator/0.log"
Jul 30 01:49:02.159333 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:49:02.159302    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-x85f9_24e359fe-816f-4977-a4e6-3d3bb82a1634/graceful-termination/0.log"
Jul 30 01:49:03.476992 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:49:03.476956    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lnhk8_e419b701-ba27-4810-9851-ae5242cdc2c5/kube-multus-additional-cni-plugins/0.log"
Jul 30 01:49:03.498465 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:49:03.498439    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lnhk8_e419b701-ba27-4810-9851-ae5242cdc2c5/egress-router-binary-copy/0.log"
Jul 30 01:49:03.517171 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:49:03.517145    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lnhk8_e419b701-ba27-4810-9851-ae5242cdc2c5/cni-plugins/0.log"
Jul 30 01:49:03.535746 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:49:03.535715    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lnhk8_e419b701-ba27-4810-9851-ae5242cdc2c5/bond-cni-plugin/0.log"
Jul 30 01:49:03.554614 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:49:03.554586    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lnhk8_e419b701-ba27-4810-9851-ae5242cdc2c5/routeoverride-cni/0.log"
Jul 30 01:49:03.573666 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:49:03.573639    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lnhk8_e419b701-ba27-4810-9851-ae5242cdc2c5/whereabouts-cni-bincopy/0.log"
Jul 30 01:49:03.593800 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:49:03.593697    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lnhk8_e419b701-ba27-4810-9851-ae5242cdc2c5/whereabouts-cni/0.log"
Jul 30 01:49:03.955179 ip-10-0-141-85 kubenswrapper[2546]: I0730 01:49:03.955113    2546 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-bwfnn_9fd5e9cd-5cde-452d-b349-31a4c1a12ec9/kube-multus/0.log"