Jul 26 17:49:03.218437 ip-10-0-128-45 systemd[1]: Starting Kubernetes Kubelet...
Jul 26 17:49:03.657269 ip-10-0-128-45 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 26 17:49:03.657269 ip-10-0-128-45 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 26 17:49:03.657269 ip-10-0-128-45 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 26 17:49:03.657269 ip-10-0-128-45 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 26 17:49:03.657269 ip-10-0-128-45 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 26 17:49:03.659214 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.659107    2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 26 17:49:03.662799 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662780    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 17:49:03.662799 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662798    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 17:49:03.662799 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662802    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 17:49:03.662892 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662807    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 17:49:03.662892 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662810    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 17:49:03.662892 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662814    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 17:49:03.662892 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662818    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 17:49:03.662892 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662821    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 17:49:03.662892 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662823    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 17:49:03.662892 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662827    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 17:49:03.662892 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662830    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 17:49:03.662892 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662833    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 17:49:03.662892 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662836    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 17:49:03.662892 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662839    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 17:49:03.662892 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662842    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 17:49:03.662892 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662845    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 17:49:03.662892 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662847    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 17:49:03.662892 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662850    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 17:49:03.662892 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662853    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 17:49:03.662892 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662855    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 17:49:03.662892 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662858    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 17:49:03.662892 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662866    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 17:49:03.663418 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662869    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 17:49:03.663418 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662872    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 17:49:03.663418 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662875    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 17:49:03.663418 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662878    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 17:49:03.663418 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662881    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 17:49:03.663418 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662885    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 17:49:03.663418 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662887    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 17:49:03.663418 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662890    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 17:49:03.663418 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662892    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 17:49:03.663418 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662895    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 17:49:03.663418 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662898    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 17:49:03.663418 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662901    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 17:49:03.663418 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662903    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 17:49:03.663418 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662906    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 17:49:03.663418 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662909    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 17:49:03.663418 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662911    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 17:49:03.663418 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662914    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 17:49:03.663418 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662916    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 17:49:03.663418 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662918    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 17:49:03.663418 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662921    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 17:49:03.663951 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662923    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 17:49:03.663951 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662926    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 17:49:03.663951 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662928    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 17:49:03.663951 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662931    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 17:49:03.663951 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662933    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 17:49:03.663951 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662936    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 17:49:03.663951 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662938    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 17:49:03.663951 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662942    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 17:49:03.663951 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662946    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 17:49:03.663951 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662949    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 17:49:03.663951 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662952    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 17:49:03.663951 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662955    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 17:49:03.663951 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662959    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 17:49:03.663951 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662962    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 17:49:03.663951 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662965    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 17:49:03.663951 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662968    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 17:49:03.663951 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662970    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 17:49:03.663951 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662973    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 17:49:03.663951 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662975    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 17:49:03.663951 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662978    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 17:49:03.664477 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662982    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 17:49:03.664477 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662992    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 17:49:03.664477 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662995    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 17:49:03.664477 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.662998    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 17:49:03.664477 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663000    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 17:49:03.664477 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663003    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 17:49:03.664477 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663005    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 17:49:03.664477 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663008    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 17:49:03.664477 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663011    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 17:49:03.664477 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663013    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 17:49:03.664477 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663016    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 17:49:03.664477 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663019    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 17:49:03.664477 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663021    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 17:49:03.664477 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663024    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 17:49:03.664477 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663027    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 17:49:03.664477 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663029    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 17:49:03.664477 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663032    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 17:49:03.664477 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663035    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 17:49:03.664477 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663039    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 17:49:03.664477 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663041    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 17:49:03.665208 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663044    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 17:49:03.665208 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663048    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 17:49:03.665208 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663052    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 17:49:03.665208 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663055    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 17:49:03.665208 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663058    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 17:49:03.665208 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663108    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 17:49:03.665208 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663181    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 17:49:03.665208 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663186    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 17:49:03.665208 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663189    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 17:49:03.665208 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663192    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 17:49:03.665208 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663195    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 17:49:03.665208 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663198    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 17:49:03.665208 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663201    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 17:49:03.665208 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663204    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 17:49:03.665208 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663208    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 17:49:03.665208 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663211    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 17:49:03.665208 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.663214    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 17:49:03.666148 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666131    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 17:49:03.666148 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666147    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 17:49:03.666282 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666153    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 17:49:03.666282 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666158    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 17:49:03.666282 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666164    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 17:49:03.666282 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666172    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 17:49:03.666282 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666178    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 17:49:03.666282 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666183    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 17:49:03.666282 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666187    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 17:49:03.666282 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666192    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 17:49:03.666282 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666196    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 17:49:03.666282 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666201    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 17:49:03.666282 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666206    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 17:49:03.666282 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666210    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 17:49:03.666282 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666215    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 17:49:03.666282 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666219    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 17:49:03.666282 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666223    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 17:49:03.666282 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666228    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 17:49:03.666282 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666232    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 17:49:03.666282 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666236    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 17:49:03.667039 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666242    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 17:49:03.667039 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666248    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 17:49:03.667039 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666253    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 17:49:03.667039 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666257    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 17:49:03.667039 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666261    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 17:49:03.667039 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666265    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 17:49:03.667039 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666270    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 17:49:03.667039 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666274    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 17:49:03.667039 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666278    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 17:49:03.667039 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666282    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 17:49:03.667039 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666287    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 17:49:03.667039 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666291    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 17:49:03.667039 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666295    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 17:49:03.667039 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666299    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 17:49:03.667039 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666304    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 17:49:03.667039 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666308    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 17:49:03.667039 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666312    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 17:49:03.667039 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666316    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 17:49:03.667039 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666320    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 17:49:03.667039 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666324    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 17:49:03.667918 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666328    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 17:49:03.667918 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666332    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 17:49:03.667918 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666336    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 17:49:03.667918 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666343    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 17:49:03.667918 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666350    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 17:49:03.667918 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666354    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 17:49:03.667918 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666360    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 17:49:03.667918 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666366    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 17:49:03.667918 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666371    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 17:49:03.667918 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666375    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 17:49:03.667918 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666380    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 17:49:03.667918 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666384    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 17:49:03.667918 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666388    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 17:49:03.667918 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666393    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 17:49:03.667918 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666397    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 17:49:03.667918 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666401    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 17:49:03.667918 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666405    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 17:49:03.667918 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666410    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 17:49:03.667918 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666415    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 17:49:03.667918 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666420    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 17:49:03.668800 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666424    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 17:49:03.668800 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666428    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 17:49:03.668800 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666433    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 17:49:03.668800 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666437    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 17:49:03.668800 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666441    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 17:49:03.668800 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666445    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 17:49:03.668800 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666449    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 17:49:03.668800 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666454    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 17:49:03.668800 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666458    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 17:49:03.668800 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666462    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 17:49:03.668800 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666466    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 17:49:03.668800 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666472    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 17:49:03.668800 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666477    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 17:49:03.668800 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666481    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 17:49:03.668800 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666485    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 17:49:03.668800 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666489    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 17:49:03.668800 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666493    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 17:49:03.668800 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666497    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 17:49:03.668800 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666502    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 17:49:03.668800 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666505    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 17:49:03.669576 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666510    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 17:49:03.669576 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666514    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 17:49:03.669576 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666519    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 17:49:03.669576 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666523    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 17:49:03.669576 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666527    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 17:49:03.669576 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666532    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 17:49:03.669576 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666536    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 17:49:03.669576 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666540    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 17:49:03.669576 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666545    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 17:49:03.669576 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666550    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 17:49:03.669576 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666554    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 17:49:03.669576 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666558    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 17:49:03.669576 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666563    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 17:49:03.669576 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666568    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 17:49:03.669576 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666572    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 17:49:03.669576 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666576    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 17:49:03.669576 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666581    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 17:49:03.669576 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666585    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 17:49:03.669576 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.666589    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 17:49:03.669576 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667297    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 17:49:03.670086 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667308    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 17:49:03.670086 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667314    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 17:49:03.670086 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667319    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 17:49:03.670086 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667324    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 17:49:03.670086 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667329    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 17:49:03.670086 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667333    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 17:49:03.670086 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667338    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 17:49:03.670086 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667342    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 17:49:03.670086 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667348    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 17:49:03.670086 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667355    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 17:49:03.670086 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667359    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 17:49:03.670086 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667364    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 17:49:03.670086 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667369    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 17:49:03.670086 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667373    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 17:49:03.670086 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667377    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 17:49:03.670086 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667381    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 17:49:03.670086 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667385    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 17:49:03.670086 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667390    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 17:49:03.670086 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667394    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 17:49:03.670665 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667399    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 17:49:03.670665 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667403    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 17:49:03.670665 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667408    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 17:49:03.670665 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667412    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 17:49:03.670665 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667416    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 17:49:03.670665 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667421    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 17:49:03.670665 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667425    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 17:49:03.670665 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667429    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 17:49:03.670665 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667434    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 17:49:03.670665 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667438    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 17:49:03.670665 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667442    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 17:49:03.670665 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667446    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 17:49:03.670665 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667450    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 17:49:03.670665 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667455    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 17:49:03.670665 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667461    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 17:49:03.670665 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667466    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 17:49:03.670665 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667471    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 17:49:03.670665 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667477    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 17:49:03.670665 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667482    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 17:49:03.671170 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667486    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 17:49:03.671170 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667491    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 17:49:03.671170 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667496    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 17:49:03.671170 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667500    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 17:49:03.671170 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667506    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 17:49:03.671170 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667511    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 17:49:03.671170 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667515    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 17:49:03.671170 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667520    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 17:49:03.671170 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667524    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 17:49:03.671170 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667528    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 17:49:03.671170 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667533    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 17:49:03.671170 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667537    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 17:49:03.671170 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667542    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 17:49:03.671170 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667546    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 17:49:03.671170 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667550    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 17:49:03.671170 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667555    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 17:49:03.671170 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667560    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 17:49:03.671170 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667564    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 17:49:03.671170 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667568    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 17:49:03.671170 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667573    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 17:49:03.671882 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667577    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 17:49:03.671882 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667581    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 17:49:03.671882 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667585    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 17:49:03.671882 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667589    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 17:49:03.671882 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667593    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 17:49:03.671882 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667597    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 17:49:03.671882 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667602    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 17:49:03.671882 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667606    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 17:49:03.671882 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667610    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 17:49:03.671882 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667614    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 17:49:03.671882 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667621    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 17:49:03.671882 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667628    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 17:49:03.671882 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667633    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 17:49:03.671882 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667637    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 17:49:03.671882 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667642    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 17:49:03.671882 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667646    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 17:49:03.671882 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667651    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 17:49:03.671882 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667655    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 17:49:03.671882 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667659    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 17:49:03.671882 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667663    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 17:49:03.672386 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667667    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 17:49:03.672386 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667672    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 17:49:03.672386 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667676    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 17:49:03.672386 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667681    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 17:49:03.672386 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667685    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 17:49:03.672386 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667690    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 17:49:03.672386 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667694    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 17:49:03.672386 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667699    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 17:49:03.672386 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667705    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 17:49:03.672386 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667710    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 17:49:03.672386 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667714    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 17:49:03.672386 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667719    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 17:49:03.672386 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667723    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 17:49:03.672386 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667728    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 17:49:03.672386 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667732    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 17:49:03.672386 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667736    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 17:49:03.672386 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667741    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 17:49:03.672386 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667745    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 17:49:03.672386 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667749    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 17:49:03.672386 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.667753    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 17:49:03.672979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.667853    2570 flags.go:64] FLAG: --address="0.0.0.0"
Jul 26 17:49:03.672979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.667870    2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 26 17:49:03.672979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.667879    2570 flags.go:64] FLAG: --anonymous-auth="true"
Jul 26 17:49:03.672979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.667886    2570 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 26 17:49:03.672979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.667894    2570 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 26 17:49:03.672979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.667900    2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 26 17:49:03.672979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.667906    2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 26 17:49:03.672979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.667913    2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 26 17:49:03.672979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.667918    2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 26 17:49:03.672979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.667923    2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 26 17:49:03.672979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.667928    2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 26 17:49:03.672979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.667933    2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 26 17:49:03.672979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.667938    2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 26 17:49:03.672979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.667942    2570 flags.go:64] FLAG: --cgroup-root=""
Jul 26 17:49:03.672979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.667947    2570 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 26 17:49:03.672979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.667952    2570 flags.go:64] FLAG: --client-ca-file=""
Jul 26 17:49:03.672979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.667957    2570 flags.go:64] FLAG: --cloud-provider="external"
Jul 26 17:49:03.672979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.667961    2570 flags.go:64] FLAG: --cluster-dns="[]"
Jul 26 17:49:03.672979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.667969    2570 flags.go:64] FLAG: --cluster-domain=""
Jul 26 17:49:03.672979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.667975    2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 26 17:49:03.672979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.667980    2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 26 17:49:03.672979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.667985    2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 26 17:49:03.672979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.667991    2570 flags.go:64] FLAG: --container-log-max-files="5"
Jul 26 17:49:03.672979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668001    2570 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668006    2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668012    2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668017    2570 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668022    2570 flags.go:64] FLAG: --contention-profiling="false"
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668026    2570 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668031    2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668036    2570 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668041    2570 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668048    2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668053    2570 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668058    2570 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668062    2570 flags.go:64] FLAG: --enable-load-reader="false"
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668067    2570 flags.go:64] FLAG: --enable-server="true"
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668072    2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668080    2570 flags.go:64] FLAG: --event-burst="100"
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668085    2570 flags.go:64] FLAG: --event-qps="50"
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668090    2570 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668094    2570 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668099    2570 flags.go:64] FLAG: --eviction-hard=""
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668105    2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668127    2570 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668132    2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668137    2570 flags.go:64] FLAG: --eviction-soft=""
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668142    2570 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 26 17:49:03.673585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668147    2570 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 26 17:49:03.674213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668152    2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 26 17:49:03.674213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668157    2570 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 26 17:49:03.674213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668161    2570 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 26 17:49:03.674213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668166    2570 flags.go:64] FLAG: --fail-swap-on="true"
Jul 26 17:49:03.674213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668171    2570 flags.go:64] FLAG: --feature-gates=""
Jul 26 17:49:03.674213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668177    2570 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 26 17:49:03.674213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668182    2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 26 17:49:03.674213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668188    2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 26 17:49:03.674213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668193    2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 26 17:49:03.674213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668198    2570 flags.go:64] FLAG: --healthz-port="10248"
Jul 26 17:49:03.674213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668203    2570 flags.go:64] FLAG: --help="false"
Jul 26 17:49:03.674213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668208    2570 flags.go:64] FLAG: --hostname-override="ip-10-0-128-45.ec2.internal"
Jul 26 17:49:03.674213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668213    2570 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 26 17:49:03.674213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668217    2570 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 26 17:49:03.674213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668222    2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 26 17:49:03.674213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668227    2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 26 17:49:03.674213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668234    2570 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 26 17:49:03.674213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668238    2570 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 26 17:49:03.674213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668243    2570 flags.go:64] FLAG: --image-service-endpoint=""
Jul 26 17:49:03.674213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668248    2570 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 26 17:49:03.674213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668253    2570 flags.go:64] FLAG: --kube-api-burst="100"
Jul 26 17:49:03.674213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668258    2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 26 17:49:03.674213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668264    2570 flags.go:64] FLAG: --kube-api-qps="50"
Jul 26 17:49:03.674213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668268    2570 flags.go:64] FLAG: --kube-reserved=""
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668273    2570 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668278    2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668283    2570 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668288    2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668293    2570 flags.go:64] FLAG: --lock-file=""
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668297    2570 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668302    2570 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668307    2570 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668316    2570 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668320    2570 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668325    2570 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668331    2570 flags.go:64] FLAG: --logging-format="text"
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668336    2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668341    2570 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668346    2570 flags.go:64] FLAG: --manifest-url=""
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668350    2570 flags.go:64] FLAG: --manifest-url-header=""
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668357    2570 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668362    2570 flags.go:64] FLAG: --max-open-files="1000000"
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668369    2570 flags.go:64] FLAG: --max-pods="110"
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668373    2570 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668378    2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668383    2570 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668387    2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668392    2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 26 17:49:03.674812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668397    2570 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668402    2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668414    2570 flags.go:64] FLAG: --node-status-max-images="50"
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668418    2570 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668423    2570 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668428    2570 flags.go:64] FLAG: --pod-cidr=""
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668434    2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668443    2570 flags.go:64] FLAG: --pod-manifest-path=""
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668448    2570 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668453    2570 flags.go:64] FLAG: --pods-per-core="0"
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668457    2570 flags.go:64] FLAG: --port="10250"
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668462    2570 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668467    2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08a143483d69eda7b"
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668472    2570 flags.go:64] FLAG: --qos-reserved=""
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668477    2570 flags.go:64] FLAG: --read-only-port="10255"
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668482    2570 flags.go:64] FLAG: --register-node="true"
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668487    2570 flags.go:64] FLAG: --register-with-taints=""
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668492    2570 flags.go:64] FLAG: --registry-burst="10"
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668497    2570 flags.go:64] FLAG: --registry-qps="5"
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668501    2570 flags.go:64] FLAG: --reserved-cpus=""
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668506    2570 flags.go:64] FLAG: --reserved-memory=""
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668512    2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668518    2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668523    2570 flags.go:64] FLAG: --rotate-certificates="false"
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668527    2570 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 26 17:49:03.675457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668532    2570 flags.go:64] FLAG: --runonce="false"
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668537    2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668542    2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668547    2570 flags.go:64] FLAG: --seccomp-default="false"
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668551    2570 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668556    2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668561    2570 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668566    2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668571    2570 flags.go:64] FLAG: --storage-driver-password="root"
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668576    2570 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668581    2570 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668585    2570 flags.go:64] FLAG: --storage-driver-user="root"
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668590    2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668595    2570 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668600    2570 flags.go:64] FLAG: --system-cgroups=""
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668606    2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668614    2570 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668619    2570 flags.go:64] FLAG: --tls-cert-file=""
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668624    2570 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668631    2570 flags.go:64] FLAG: --tls-min-version=""
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668638    2570 flags.go:64] FLAG: --tls-private-key-file=""
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668643    2570 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668648    2570 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668653    2570 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668658    2570 flags.go:64] FLAG: --v="2"
Jul 26 17:49:03.676153 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668664    2570 flags.go:64] FLAG: --version="false"
Jul 26 17:49:03.676795 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668672    2570 flags.go:64] FLAG: --vmodule=""
Jul 26 17:49:03.676795 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668678    2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 26 17:49:03.676795 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.668683    2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 26 17:49:03.676795 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.668888    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 17:49:03.676795 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.668896    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 17:49:03.676795 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.668901    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 17:49:03.676795 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.668906    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 17:49:03.676795 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.668910    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 17:49:03.676795 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.668915    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 17:49:03.676795 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.668922    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 17:49:03.676795 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.668928    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 17:49:03.676795 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.668932    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 17:49:03.676795 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.668937    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 17:49:03.676795 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.668941    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 17:49:03.676795 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.668946    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 17:49:03.676795 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.668950    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 17:49:03.676795 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.668955    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 17:49:03.676795 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.668960    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 17:49:03.676795 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.668964    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 17:49:03.676795 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.668969    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 17:49:03.677307 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.668973    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 17:49:03.677307 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.668980    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 17:49:03.677307 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.668985    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 17:49:03.677307 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.668989    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 17:49:03.677307 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.668993    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 17:49:03.677307 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.668997    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 17:49:03.677307 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669003    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 17:49:03.677307 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669007    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 17:49:03.677307 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669011    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 17:49:03.677307 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669015    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 17:49:03.677307 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669020    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 17:49:03.677307 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669024    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 17:49:03.677307 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669028    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 17:49:03.677307 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669032    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 17:49:03.677307 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669036    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 17:49:03.677307 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669040    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 17:49:03.677307 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669044    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 17:49:03.677307 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669048    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 17:49:03.677307 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669053    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 17:49:03.677307 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669057    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 17:49:03.677793 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669061    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 17:49:03.677793 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669066    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 17:49:03.677793 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669070    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 17:49:03.677793 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669074    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 17:49:03.677793 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669077    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 17:49:03.677793 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669082    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 17:49:03.677793 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669086    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 17:49:03.677793 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669090    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 17:49:03.677793 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669094    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 17:49:03.677793 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669098    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 17:49:03.677793 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669103    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 17:49:03.677793 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669107    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 17:49:03.677793 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669127    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 17:49:03.677793 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669134    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 17:49:03.677793 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669139    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 17:49:03.677793 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669143    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 17:49:03.677793 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669147    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 17:49:03.677793 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669152    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 17:49:03.677793 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669158    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 17:49:03.677793 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669163    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 17:49:03.678304 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669167    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 17:49:03.678304 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669171    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 17:49:03.678304 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669175    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 17:49:03.678304 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669180    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 17:49:03.678304 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669184    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 17:49:03.678304 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669188    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 17:49:03.678304 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669192    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 17:49:03.678304 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669196    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 17:49:03.678304 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669203    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 17:49:03.678304 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669209    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 17:49:03.678304 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669214    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 17:49:03.678304 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669219    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 17:49:03.678304 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669223    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 17:49:03.678304 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669227    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 17:49:03.678304 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669232    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 17:49:03.678304 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669235    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 17:49:03.678304 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669240    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 17:49:03.678304 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669244    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 17:49:03.678304 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669249    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 17:49:03.678304 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669253    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 17:49:03.678801 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669257    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 17:49:03.678801 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669261    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 17:49:03.678801 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669265    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 17:49:03.678801 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669269    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 17:49:03.678801 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669273    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 17:49:03.678801 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669279    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 17:49:03.678801 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669284    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 17:49:03.678801 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669289    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 17:49:03.678801 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669295    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 17:49:03.678801 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669302    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 17:49:03.678801 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669309    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 17:49:03.678801 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669313    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 17:49:03.678801 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669317    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 17:49:03.678801 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669321    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 17:49:03.678801 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669327    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 17:49:03.678801 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669331    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 17:49:03.678801 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669335    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 17:49:03.678801 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669339    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 17:49:03.678801 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669343    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 17:49:03.679335 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669347    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 17:49:03.679335 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669352    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 17:49:03.679335 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.669356    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 17:49:03.679335 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.670365    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 26 17:49:03.679335 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.677438    2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 26 17:49:03.679335 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.677453    2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 26 17:49:03.679335 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.677474    2570 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 26 17:49:03.679335 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.677479    2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 26 17:49:03.679335 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677536    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 17:49:03.679335 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677549    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 17:49:03.679335 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677553    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 17:49:03.679335 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677556    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 17:49:03.679335 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677559    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 17:49:03.679335 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677562    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 17:49:03.679335 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677565    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 17:49:03.679705 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677568    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 17:49:03.679705 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677570    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 17:49:03.679705 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677574    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 17:49:03.679705 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677577    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 17:49:03.679705 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677579    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 17:49:03.679705 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677583    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 17:49:03.679705 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677586    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 17:49:03.679705 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677589    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 17:49:03.679705 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677592    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 17:49:03.679705 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677594    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 17:49:03.679705 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677597    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 17:49:03.679705 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677599    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 17:49:03.679705 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677602    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 17:49:03.679705 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677604    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 17:49:03.679705 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677607    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 17:49:03.679705 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677609    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 17:49:03.679705 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677612    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 17:49:03.679705 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677615    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 17:49:03.679705 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677617    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 17:49:03.679705 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677620    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 17:49:03.680186 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677622    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 17:49:03.680186 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677625    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 17:49:03.680186 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677629    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 17:49:03.680186 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677631    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 17:49:03.680186 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677634    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 17:49:03.680186 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677637    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 17:49:03.680186 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677639    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 17:49:03.680186 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677642    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 17:49:03.680186 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677644    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 17:49:03.680186 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677647    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 17:49:03.680186 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677649    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 17:49:03.680186 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677652    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 17:49:03.680186 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677654    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 17:49:03.680186 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677657    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 17:49:03.680186 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677660    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 17:49:03.680186 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677663    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 17:49:03.680186 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677665    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 17:49:03.680186 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677668    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 17:49:03.680186 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677670    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 17:49:03.680186 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677674    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 17:49:03.680671 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677678    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 17:49:03.680671 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677681    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 17:49:03.680671 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677683    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 17:49:03.680671 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677685    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 17:49:03.680671 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677688    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 17:49:03.680671 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677691    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 17:49:03.680671 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677693    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 17:49:03.680671 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677696    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 17:49:03.680671 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677698    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 17:49:03.680671 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677701    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 17:49:03.680671 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677704    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 17:49:03.680671 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677706    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 17:49:03.680671 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677709    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 17:49:03.680671 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677711    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 17:49:03.680671 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677714    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 17:49:03.680671 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677717    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 17:49:03.680671 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677719    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 17:49:03.680671 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677723    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 17:49:03.680671 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677727    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 17:49:03.681147 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677730    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 17:49:03.681147 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677733    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 17:49:03.681147 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677736    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 17:49:03.681147 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677740    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 17:49:03.681147 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677743    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 17:49:03.681147 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677746    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 17:49:03.681147 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677749    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 17:49:03.681147 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677752    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 17:49:03.681147 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677754    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 17:49:03.681147 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677757    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 17:49:03.681147 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677760    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 17:49:03.681147 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677762    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 17:49:03.681147 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677766    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 17:49:03.681147 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677769    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 17:49:03.681147 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677771    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 17:49:03.681147 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677774    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 17:49:03.681147 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677777    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 17:49:03.681147 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677780    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 17:49:03.681147 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677782    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 17:49:03.681147 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677797    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 17:49:03.681646 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677801    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 17:49:03.681646 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677805    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 17:49:03.681646 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677808    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 17:49:03.681646 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677811    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 17:49:03.681646 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677813    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 17:49:03.681646 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677816    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 17:49:03.681646 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677818    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 17:49:03.681646 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677821    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 17:49:03.681646 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677824    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 17:49:03.681646 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677826    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 17:49:03.681646 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677829    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 17:49:03.681646 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677832    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 17:49:03.681646 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677834    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 17:49:03.681646 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.677839    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 26 17:49:03.681646 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677952    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 17:49:03.682004 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677956    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 17:49:03.682004 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677959    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 17:49:03.682004 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677962    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 17:49:03.682004 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677964    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 17:49:03.682004 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677967    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 17:49:03.682004 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677969    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 17:49:03.682004 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677972    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 17:49:03.682004 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677975    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 17:49:03.682004 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677978    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 17:49:03.682004 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677980    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 17:49:03.682004 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677984    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 17:49:03.682004 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677986    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 17:49:03.682004 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677989    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 17:49:03.682004 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677992    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 17:49:03.682004 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677994    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 17:49:03.682004 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677997    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 17:49:03.682004 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.677999    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 17:49:03.682004 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678002    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 17:49:03.682004 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678006    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 17:49:03.682475 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678009    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 17:49:03.682475 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678012    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 17:49:03.682475 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678015    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 17:49:03.682475 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678017    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 17:49:03.682475 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678020    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 17:49:03.682475 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678023    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 17:49:03.682475 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678025    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 17:49:03.682475 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678028    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 17:49:03.682475 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678030    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 17:49:03.682475 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678033    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 17:49:03.682475 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678035    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 17:49:03.682475 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678038    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 17:49:03.682475 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678040    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 17:49:03.682475 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678043    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 17:49:03.682475 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678045    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 17:49:03.682475 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678048    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 17:49:03.682475 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678050    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 17:49:03.682475 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678053    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 17:49:03.682475 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678055    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 17:49:03.682475 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678058    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 17:49:03.682960 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678060    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 17:49:03.682960 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678063    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 17:49:03.682960 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678065    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 17:49:03.682960 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678067    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 17:49:03.682960 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678071    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 17:49:03.682960 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678073    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 17:49:03.682960 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678076    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 17:49:03.682960 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678078    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 17:49:03.682960 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678081    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 17:49:03.682960 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678084    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 17:49:03.682960 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678086    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 17:49:03.682960 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678088    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 17:49:03.682960 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678092    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 17:49:03.682960 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678095    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 17:49:03.682960 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678098    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 17:49:03.682960 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678101    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 17:49:03.682960 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678104    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 17:49:03.682960 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678107    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 17:49:03.682960 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678125    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 17:49:03.683428 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678128    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 17:49:03.683428 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678131    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 17:49:03.683428 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678134    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 17:49:03.683428 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678136    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 17:49:03.683428 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678139    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 17:49:03.683428 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678142    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 17:49:03.683428 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678145    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 17:49:03.683428 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678147    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 17:49:03.683428 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678150    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 17:49:03.683428 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678152    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 17:49:03.683428 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678155    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 17:49:03.683428 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678159    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 17:49:03.683428 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678162    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 17:49:03.683428 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678165    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 17:49:03.683428 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678167    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 17:49:03.683428 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678170    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 17:49:03.683428 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678172    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 17:49:03.683428 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678175    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 17:49:03.683428 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678178    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 17:49:03.683428 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678180    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 17:49:03.683899 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678183    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 17:49:03.683899 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678185    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 17:49:03.683899 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678188    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 17:49:03.683899 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678190    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 17:49:03.683899 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678193    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 17:49:03.683899 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678195    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 17:49:03.683899 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678198    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 17:49:03.683899 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678200    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 17:49:03.683899 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678203    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 17:49:03.683899 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678205    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 17:49:03.683899 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678208    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 17:49:03.683899 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678210    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 17:49:03.683899 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678213    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 17:49:03.683899 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678215    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 17:49:03.683899 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678218    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 17:49:03.683899 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678220    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 17:49:03.683899 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678223    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 17:49:03.683899 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678225    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 17:49:03.683899 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678227    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 17:49:03.683899 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:03.678230    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 17:49:03.684398 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.678235    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 26 17:49:03.684398 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.678335    2570 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 26 17:49:03.684398 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.680889    2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 26 17:49:03.684398 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.682253    2570 server.go:1019] "Starting client certificate rotation"
Jul 26 17:49:03.684398 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.682352    2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 26 17:49:03.684398 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.682385    2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 26 17:49:03.685608 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.685594    2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 26 17:49:03.692928 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.692908    2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 26 17:49:03.710667 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.710641    2570 log.go:25] "Validated CRI v1 runtime API"
Jul 26 17:49:03.712731 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.712714    2570 log.go:25] "Validated CRI v1 image API"
Jul 26 17:49:03.713512 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.713485    2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 26 17:49:03.713512 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.713494    2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 26 17:49:03.715857 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.715839    2570 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8dc30d32-ce76-4de0-813c-aa76c255f97f:/dev/nvme0n1p4 f459fd56-b5c6-490d-b949-97f696575eef:/dev/nvme0n1p3]
Jul 26 17:49:03.715903 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.715857    2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 26 17:49:03.722899 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.722779    2570 manager.go:216] Machine: {Timestamp:2026-07-26 17:49:03.721442515 +0000 UTC m=+0.373633363 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3105327 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29743145d5598fecc8dd2b3f0143d3 SystemUUID:ec297431-45d5-598f-ecc8-dd2b3f0143d3 BootID:8ebcdf0d-26cf-4475-a28a-d79be36f3850 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:40:7a:3b:c1:f1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:40:7a:3b:c1:f1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:41:eb:f9:be:32 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 26 17:49:03.722899 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.722891    2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 26 17:49:03.723011 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.722979    2570 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 26 17:49:03.726313 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.726283    2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 26 17:49:03.726460 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.726315    2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-45.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 26 17:49:03.727516 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.727505    2570 topology_manager.go:138] "Creating topology manager with none policy"
Jul 26 17:49:03.727561 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.727519    2570 container_manager_linux.go:309] "Creating device plugin manager"
Jul 26 17:49:03.727561 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.727527    2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 26 17:49:03.728972 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.728960    2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 26 17:49:03.730314 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.730303    2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 26 17:49:03.732916 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.732904    2570 state_mem.go:36] "Initialized new in-memory state store"
Jul 26 17:49:03.733031 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.733021    2570 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 26 17:49:03.733062 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.733052    2570 kubelet.go:486] "Attempting to sync node with API server"
Jul 26 17:49:03.733092 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.733062    2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 26 17:49:03.733092 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.733074    2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 26 17:49:03.733092 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.733084    2570 kubelet.go:398] "Adding apiserver pod source"
Jul 26 17:49:03.733220 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.733093    2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 26 17:49:03.733564 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.733552    2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 26 17:49:03.733643 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.733571    2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 26 17:49:03.735792 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.735776    2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 26 17:49:03.738190 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.738139    2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 26 17:49:03.738190 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.738169    2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 26 17:49:03.740004 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.739993    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 26 17:49:03.740054 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.740009    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 26 17:49:03.740054 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.740015    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 26 17:49:03.740054 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.740020    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 26 17:49:03.740054 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.740029    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 26 17:49:03.740054 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.740035    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 26 17:49:03.740054 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.740046    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 26 17:49:03.740054 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.740051    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 26 17:49:03.740054 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.740057    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 26 17:49:03.740280 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.740063    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 26 17:49:03.740280 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.740073    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 26 17:49:03.740280 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.740078    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 26 17:49:03.743054 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.743041    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 26 17:49:03.743054 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.743054    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 26 17:49:03.743630 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:03.743584    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-45.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 26 17:49:03.743687 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:03.743587    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 26 17:49:03.746261 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.746249    2570 server.go:1268] "Started kubelet"
Jul 26 17:49:03.746361 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.746335    2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 26 17:49:03.746407 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.746353    2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 26 17:49:03.746460 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.746413    2570 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 26 17:49:03.747156 ip-10-0-128-45 systemd[1]: Started Kubernetes Kubelet.
Jul 26 17:49:03.749561 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.748731    2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 26 17:49:03.751477 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.751458    2570 server.go:310] "Adding debug handlers to kubelet server"
Jul 26 17:49:03.754231 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:03.754208    2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 26 17:49:03.756026 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.756011    2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-45.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 26 17:49:03.756937 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:03.755886    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-45.ec2.internal.18c5e8357cc14c38  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-45.ec2.internal,UID:ip-10-0-128-45.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-45.ec2.internal,},FirstTimestamp:2026-07-26 17:49:03.74622316 +0000 UTC m=+0.398414009,LastTimestamp:2026-07-26 17:49:03.74622316 +0000 UTC m=+0.398414009,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-45.ec2.internal,}"
Jul 26 17:49:03.761127 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.761101    2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 26 17:49:03.761190 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.761122    2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 26 17:49:03.761390 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:03.761240    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-45.ec2.internal\" not found"
Jul 26 17:49:03.761897 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.761874    2570 factory.go:55] Registering systemd factory
Jul 26 17:49:03.761897 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.761892    2570 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 26 17:49:03.762030 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.761909    2570 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 26 17:49:03.762030 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.761896    2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 26 17:49:03.762030 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.761898    2570 factory.go:223] Registration of the systemd container factory successfully
Jul 26 17:49:03.762030 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.761996    2570 reconstruct.go:97] "Volume reconstruction finished"
Jul 26 17:49:03.762030 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.762004    2570 reconciler.go:29] "Reconciler: start to sync state"
Jul 26 17:49:03.762250 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.762179    2570 factory.go:153] Registering CRI-O factory
Jul 26 17:49:03.762250 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.762193    2570 factory.go:223] Registration of the crio container factory successfully
Jul 26 17:49:03.762250 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.762246    2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 26 17:49:03.762361 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.762271    2570 factory.go:103] Registering Raw factory
Jul 26 17:49:03.762361 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.762286    2570 manager.go:1196] Started watching for new ooms in manager
Jul 26 17:49:03.762662 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.762639    2570 manager.go:349] Starting recovery of all containers
Jul 26 17:49:03.762862 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:03.762837    2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-45.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 26 17:49:03.763047 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:03.763024    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 26 17:49:03.768229 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.768208    2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-c6wvz"
Jul 26 17:49:03.773573 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.773557    2570 manager.go:354] Recovery completed
Jul 26 17:49:03.776923 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.776901    2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-c6wvz"
Jul 26 17:49:03.777998 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.777980    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 17:49:03.780026 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.780011    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-45.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 17:49:03.780093 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.780040    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-45.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 17:49:03.780093 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.780055    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-45.ec2.internal" event="NodeHasSufficientPID"
Jul 26 17:49:03.780635 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.780619    2570 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 26 17:49:03.780635 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.780634    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 17:49:03.780757 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.780636    2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 26 17:49:03.780757 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.780683    2570 state_mem.go:36] "Initialized new in-memory state store"
Jul 26 17:49:03.781367 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.781348    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-45.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 17:49:03.781450 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.781385    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-45.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 17:49:03.781450 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.781399    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-45.ec2.internal" event="NodeHasSufficientPID"
Jul 26 17:49:03.783348 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.783331    2570 policy_none.go:49] "None policy: Start"
Jul 26 17:49:03.783412 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.783362    2570 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 26 17:49:03.783412 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.783375    2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 26 17:49:03.785340 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.785323    2570 policy_none.go:47] "Start"
Jul 26 17:49:03.812711 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.812691    2570 manager.go:337] "Starting Device Plugin manager"
Jul 26 17:49:03.812803 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:03.812726    2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 26 17:49:03.812803 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.812737    2570 server.go:87] "Starting device plugin registration server"
Jul 26 17:49:03.813037 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.813023    2570 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 26 17:49:03.813083 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.813041    2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 26 17:49:03.813361 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.813175    2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 26 17:49:03.813361 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.813265    2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 26 17:49:03.813361 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.813274    2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 26 17:49:03.814014 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:03.813841    2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 26 17:49:03.814089 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:03.814050    2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-45.ec2.internal\" not found"
Jul 26 17:49:03.830582 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.830547    2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 26 17:49:03.842666 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.831660    2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 26 17:49:03.842666 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.831696    2570 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 26 17:49:03.842666 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.831714    2570 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 26 17:49:03.842666 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:03.831747    2570 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 26 17:49:03.842666 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.833820    2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 17:49:03.913887 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.913804    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 17:49:03.914821 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.914791    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-45.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 17:49:03.914945 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.914835    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-45.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 17:49:03.914945 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.914849    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-45.ec2.internal" event="NodeHasSufficientPID"
Jul 26 17:49:03.914945 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.914877    2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-45.ec2.internal"
Jul 26 17:49:03.924875 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.924857    2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-45.ec2.internal"
Jul 26 17:49:03.924981 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:03.924881    2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-45.ec2.internal\": node \"ip-10-0-128-45.ec2.internal\" not found"
Jul 26 17:49:03.932148 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.932097    2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-45.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-45.ec2.internal"]
Jul 26 17:49:03.932293 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.932198    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 17:49:03.933106 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.933090    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-45.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 17:49:03.933207 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.933134    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-45.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 17:49:03.933207 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.933151    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-45.ec2.internal" event="NodeHasSufficientPID"
Jul 26 17:49:03.935494 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.935481    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 17:49:03.935608 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.935594    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-45.ec2.internal"
Jul 26 17:49:03.935649 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.935642    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 17:49:03.936297 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.936283    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-45.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 17:49:03.936297 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.936291    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-45.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 17:49:03.936401 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.936310    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-45.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 17:49:03.936401 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.936313    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-45.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 17:49:03.936401 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.936322    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-45.ec2.internal" event="NodeHasSufficientPID"
Jul 26 17:49:03.936529 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.936323    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-45.ec2.internal" event="NodeHasSufficientPID"
Jul 26 17:49:03.938476 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.938461    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-45.ec2.internal"
Jul 26 17:49:03.938543 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.938490    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 17:49:03.939278 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.939260    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-45.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 17:49:03.939405 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.939293    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-45.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 17:49:03.939405 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:03.939310    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-45.ec2.internal" event="NodeHasSufficientPID"
Jul 26 17:49:03.951227 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:03.951206    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-45.ec2.internal\" not found"
Jul 26 17:49:03.961976 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:03.961952    2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-45.ec2.internal\" not found" node="ip-10-0-128-45.ec2.internal"
Jul 26 17:49:03.966579 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:03.966562    2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-45.ec2.internal\" not found" node="ip-10-0-128-45.ec2.internal"
Jul 26 17:49:04.051515 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:04.051474    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-45.ec2.internal\" not found"
Jul 26 17:49:04.062803 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:04.062773    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e256c610577a2f8cee15b21c5c7fed95-config\") pod \"kube-apiserver-proxy-ip-10-0-128-45.ec2.internal\" (UID: \"e256c610577a2f8cee15b21c5c7fed95\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-45.ec2.internal"
Jul 26 17:49:04.062803 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:04.062804    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2ae3bee3a2b4c14b81fcf318d81e0f7e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-45.ec2.internal\" (UID: \"2ae3bee3a2b4c14b81fcf318d81e0f7e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-45.ec2.internal"
Jul 26 17:49:04.062966 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:04.062831    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ae3bee3a2b4c14b81fcf318d81e0f7e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-45.ec2.internal\" (UID: \"2ae3bee3a2b4c14b81fcf318d81e0f7e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-45.ec2.internal"
Jul 26 17:49:04.152448 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:04.152405    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-45.ec2.internal\" not found"
Jul 26 17:49:04.163745 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:04.163721    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e256c610577a2f8cee15b21c5c7fed95-config\") pod \"kube-apiserver-proxy-ip-10-0-128-45.ec2.internal\" (UID: \"e256c610577a2f8cee15b21c5c7fed95\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-45.ec2.internal"
Jul 26 17:49:04.163807 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:04.163750    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2ae3bee3a2b4c14b81fcf318d81e0f7e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-45.ec2.internal\" (UID: \"2ae3bee3a2b4c14b81fcf318d81e0f7e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-45.ec2.internal"
Jul 26 17:49:04.163807 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:04.163772    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ae3bee3a2b4c14b81fcf318d81e0f7e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-45.ec2.internal\" (UID: \"2ae3bee3a2b4c14b81fcf318d81e0f7e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-45.ec2.internal"
Jul 26 17:49:04.163937 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:04.163819    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2ae3bee3a2b4c14b81fcf318d81e0f7e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-45.ec2.internal\" (UID: \"2ae3bee3a2b4c14b81fcf318d81e0f7e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-45.ec2.internal"
Jul 26 17:49:04.163937 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:04.163837    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e256c610577a2f8cee15b21c5c7fed95-config\") pod \"kube-apiserver-proxy-ip-10-0-128-45.ec2.internal\" (UID: \"e256c610577a2f8cee15b21c5c7fed95\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-45.ec2.internal"
Jul 26 17:49:04.163937 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:04.163893    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ae3bee3a2b4c14b81fcf318d81e0f7e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-45.ec2.internal\" (UID: \"2ae3bee3a2b4c14b81fcf318d81e0f7e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-45.ec2.internal"
Jul 26 17:49:04.253224 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:04.253184    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-45.ec2.internal\" not found"
Jul 26 17:49:04.265649 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:04.265620    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-45.ec2.internal"
Jul 26 17:49:04.271038 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:04.270836    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-45.ec2.internal"
Jul 26 17:49:04.354124 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:04.354066    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-45.ec2.internal\" not found"
Jul 26 17:49:04.454637 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:04.454555    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-45.ec2.internal\" not found"
Jul 26 17:49:04.556606 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:04.556564    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-45.ec2.internal\" not found"
Jul 26 17:49:04.657774 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:04.657737    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-45.ec2.internal\" not found"
Jul 26 17:49:04.682735 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:04.682706    2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 26 17:49:04.682873 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:04.682855    2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 26 17:49:04.758720 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:04.758696    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-45.ec2.internal\" not found"
Jul 26 17:49:04.783258 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:04.783213    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-25 17:44:03 +0000 UTC" deadline="2028-02-21 00:18:17.61621969 +0000 UTC"
Jul 26 17:49:04.783258 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:04.783249    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13782h29m12.832974137s"
Jul 26 17:49:04.859144 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:04.859098    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-45.ec2.internal\" not found"
Jul 26 17:49:04.939582 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:04.939536    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode256c610577a2f8cee15b21c5c7fed95.slice/crio-00c24fd9f29b763d092dc3317b240aaf83d074f1b0637317a3a7e8a9b542b995 WatchSource:0}: Error finding container 00c24fd9f29b763d092dc3317b240aaf83d074f1b0637317a3a7e8a9b542b995: Status 404 returned error can't find the container with id 00c24fd9f29b763d092dc3317b240aaf83d074f1b0637317a3a7e8a9b542b995
Jul 26 17:49:04.948832 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:04.948811    2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 17:49:04.959989 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:04.959968    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-45.ec2.internal\" not found"
Jul 26 17:49:04.962844 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:04.962821    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 17:49:04.980563 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:04.980540    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ae3bee3a2b4c14b81fcf318d81e0f7e.slice/crio-efadbc58422aceeb7372aad767330fb412f5a15037596ec2f29d11ba1c7eb93b WatchSource:0}: Error finding container efadbc58422aceeb7372aad767330fb412f5a15037596ec2f29d11ba1c7eb93b: Status 404 returned error can't find the container with id efadbc58422aceeb7372aad767330fb412f5a15037596ec2f29d11ba1c7eb93b
Jul 26 17:49:05.060786 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:05.060747    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-45.ec2.internal\" not found"
Jul 26 17:49:05.166427 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:05.166398    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-45.ec2.internal\" not found"
Jul 26 17:49:05.267144 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:05.267094    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-45.ec2.internal\" not found"
Jul 26 17:49:05.331047 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.330974    2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 17:49:05.333587 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.333568    2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 17:49:05.362753 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.362729    2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-45.ec2.internal"
Jul 26 17:49:05.373078 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.373058    2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 26 17:49:05.374368 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.374353    2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-45.ec2.internal"
Jul 26 17:49:05.386048 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.386026    2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 26 17:49:05.709578 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.709338    2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 17:49:05.734233 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.733995    2570 apiserver.go:52] "Watching apiserver"
Jul 26 17:49:05.745998 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.745965    2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 26 17:49:05.748384 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.748234    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-974ql","openshift-image-registry/node-ca-ddfsj","openshift-multus/multus-tslqc","openshift-multus/network-metrics-daemon-7mtnm","openshift-network-operator/iptables-alerter-8sbv6","openshift-ovn-kubernetes/ovnkube-node-5zh25","kube-system/global-pull-secret-syncer-96jft","kube-system/konnectivity-agent-khbbn","kube-system/kube-apiserver-proxy-ip-10-0-128-45.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c","openshift-cluster-node-tuning-operator/tuned-ccd9g","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-45.ec2.internal","openshift-multus/multus-additional-cni-plugins-s4zzb","openshift-network-diagnostics/network-check-target-h8sjb"]
Jul 26 17:49:05.749961 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.749926    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:05.750073 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:05.750034    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96jft" podUID="32cbd054-d7a7-4c00-be2e-012e16598a49"
Jul 26 17:49:05.751241 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.751218    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.754178 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.754157    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5szc2\""
Jul 26 17:49:05.754524 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.754505    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 26 17:49:05.754720 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.754704    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 26 17:49:05.754985 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.754969    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 26 17:49:05.755231 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.755216    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 26 17:49:05.756587 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.756546    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.756880 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.756858    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:05.756957 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:05.756942    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtnm" podUID="af7dc810-9c37-4155-ab59-7c4196ffb50b"
Jul 26 17:49:05.757148 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.757129    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 26 17:49:05.758904 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.758886    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8sbv6"
Jul 26 17:49:05.759354 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.759337    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.759471 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.759457    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 26 17:49:05.759534 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.759497    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-22gg5\""
Jul 26 17:49:05.760635 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.760615    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:05.760726 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:05.760688    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h8sjb" podUID="76a782a1-4344-459a-8f5c-3e84c018f0f7"
Jul 26 17:49:05.760853 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.760836    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 26 17:49:05.761214 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.761195    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 26 17:49:05.761447 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.761431    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 26 17:49:05.761697 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.761679    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ffzbq\""
Jul 26 17:49:05.761880 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.761864    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 26 17:49:05.762202 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.762184    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 26 17:49:05.762395 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.762378    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ggpm7\""
Jul 26 17:49:05.762559 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.762545    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 26 17:49:05.762728 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.762714    2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 26 17:49:05.764394 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.763592    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 26 17:49:05.764394 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.763806    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 26 17:49:05.764394 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.763972    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 26 17:49:05.764574 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.764506    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-974ql"
Jul 26 17:49:05.765685 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.765162    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ddfsj"
Jul 26 17:49:05.765785 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.765736    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c"
Jul 26 17:49:05.766445 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.766410    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 26 17:49:05.766546 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.766488    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 26 17:49:05.767194 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.766695    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-d66l6\""
Jul 26 17:49:05.767787 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.767347    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 26 17:49:05.767787 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.767379    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.767787 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.767612    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cp27q\""
Jul 26 17:49:05.768642 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.768625    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-khbbn"
Jul 26 17:49:05.770519 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.770463    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 26 17:49:05.770519 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.770510    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 26 17:49:05.770674 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.770623    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 26 17:49:05.770736 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.770699    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 26 17:49:05.770790 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.770745    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 26 17:49:05.770857 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.770841    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 26 17:49:05.771016 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.770967    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 26 17:49:05.771097 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.771068    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2x284\""
Jul 26 17:49:05.771206 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.771133    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 26 17:49:05.771291 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.771267    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cn228\""
Jul 26 17:49:05.771393 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.771337    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wfdmr\""
Jul 26 17:49:05.771448 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.771402    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 26 17:49:05.772434 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.772412    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jntqg\" (UniqueName: \"kubernetes.io/projected/76a782a1-4344-459a-8f5c-3e84c018f0f7-kube-api-access-jntqg\") pod \"network-check-target-h8sjb\" (UID: \"76a782a1-4344-459a-8f5c-3e84c018f0f7\") " pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:05.772524 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.772487    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gwl4\" (UniqueName: \"kubernetes.io/projected/b2473540-1734-4f1a-ab9a-719694dd5c96-kube-api-access-8gwl4\") pod \"aws-ebs-csi-driver-node-mbf4c\" (UID: \"b2473540-1734-4f1a-ab9a-719694dd5c96\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c"
Jul 26 17:49:05.772524 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.772514    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a337e0eb-4d13-4df5-bfe7-186de89d4dcc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s4zzb\" (UID: \"a337e0eb-4d13-4df5-bfe7-186de89d4dcc\") " pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.772627 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.772539    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/63076f7b-11ca-42d6-8d7b-2e5a96035317-cni-binary-copy\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.772627 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.772563    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-host-run-multus-certs\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.772627 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.772585    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-etc-kubernetes\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.772627 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.772607    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fcc71092-cd57-404f-aa06-8a3d8486eb33-host-slash\") pod \"iptables-alerter-8sbv6\" (UID: \"fcc71092-cd57-404f-aa06-8a3d8486eb33\") " pod="openshift-network-operator/iptables-alerter-8sbv6"
Jul 26 17:49:05.772810 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.772628    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmzxt\" (UniqueName: \"kubernetes.io/projected/1fcdab4c-350c-44c3-9bc0-a9995d33c539-kube-api-access-dmzxt\") pod \"node-resolver-974ql\" (UID: \"1fcdab4c-350c-44c3-9bc0-a9995d33c539\") " pod="openshift-dns/node-resolver-974ql"
Jul 26 17:49:05.772810 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.772655    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-hostroot\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.772810 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.772676    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-lib-modules\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.772810 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.772697    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/32cbd054-d7a7-4c00-be2e-012e16598a49-dbus\") pod \"global-pull-secret-syncer-96jft\" (UID: \"32cbd054-d7a7-4c00-be2e-012e16598a49\") " pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:05.772810 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.772719    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3f795165-d9b9-4634-a24e-d10cee628e00-ovn-node-metrics-cert\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.772810 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.772745    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a337e0eb-4d13-4df5-bfe7-186de89d4dcc-cni-binary-copy\") pod \"multus-additional-cni-plugins-s4zzb\" (UID: \"a337e0eb-4d13-4df5-bfe7-186de89d4dcc\") " pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.772810 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.772766    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-cnibin\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.772810 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.772787    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-run\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.772810 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.772808    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/048c9bc9-3ff2-4667-8392-103e4e235f0a-etc-tuned\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.773242 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.772843    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fcdab4c-350c-44c3-9bc0-a9995d33c539-tmp-dir\") pod \"node-resolver-974ql\" (UID: \"1fcdab4c-350c-44c3-9bc0-a9995d33c539\") " pod="openshift-dns/node-resolver-974ql"
Jul 26 17:49:05.773242 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.772866    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b2473540-1734-4f1a-ab9a-719694dd5c96-registration-dir\") pod \"aws-ebs-csi-driver-node-mbf4c\" (UID: \"b2473540-1734-4f1a-ab9a-719694dd5c96\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c"
Jul 26 17:49:05.773242 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.772893    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-host-run-k8s-cni-cncf-io\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.773242 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.772916    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-host-run-netns\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.773424 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773238    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af7dc810-9c37-4155-ab59-7c4196ffb50b-metrics-certs\") pod \"network-metrics-daemon-7mtnm\" (UID: \"af7dc810-9c37-4155-ab59-7c4196ffb50b\") " pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:05.773424 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773276    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-etc-sysconfig\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.773424 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773298    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-etc-kubernetes\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.773424 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773326    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1fcdab4c-350c-44c3-9bc0-a9995d33c539-hosts-file\") pod \"node-resolver-974ql\" (UID: \"1fcdab4c-350c-44c3-9bc0-a9995d33c539\") " pod="openshift-dns/node-resolver-974ql"
Jul 26 17:49:05.773424 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773350    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a337e0eb-4d13-4df5-bfe7-186de89d4dcc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s4zzb\" (UID: \"a337e0eb-4d13-4df5-bfe7-186de89d4dcc\") " pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.773424 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773375    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-system-cni-dir\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.773424 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773398    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-multus-conf-dir\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.773424 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773421    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-var-lib-kubelet\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.773784 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773444    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6v2h\" (UniqueName: \"kubernetes.io/projected/048c9bc9-3ff2-4667-8392-103e4e235f0a-kube-api-access-z6v2h\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.773784 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773467    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-systemd-units\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.773784 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773490    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-host-run-netns\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.773784 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773511    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a337e0eb-4d13-4df5-bfe7-186de89d4dcc-cnibin\") pod \"multus-additional-cni-plugins-s4zzb\" (UID: \"a337e0eb-4d13-4df5-bfe7-186de89d4dcc\") " pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.773784 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773543    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/63076f7b-11ca-42d6-8d7b-2e5a96035317-multus-daemon-config\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.773784 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773568    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z24fd\" (UniqueName: \"kubernetes.io/projected/fcc71092-cd57-404f-aa06-8a3d8486eb33-kube-api-access-z24fd\") pod \"iptables-alerter-8sbv6\" (UID: \"fcc71092-cd57-404f-aa06-8a3d8486eb33\") " pod="openshift-network-operator/iptables-alerter-8sbv6"
Jul 26 17:49:05.773784 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773591    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-var-lib-openvswitch\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.773784 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773671    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-host-run-ovn-kubernetes\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.773784 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773695    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3f795165-d9b9-4634-a24e-d10cee628e00-env-overrides\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.773784 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773718    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-run-systemd\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.773784 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773741    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-run-ovn\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.773784 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773763    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-host-cni-bin\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.773784 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773786    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.774399 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773810    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3f795165-d9b9-4634-a24e-d10cee628e00-ovnkube-script-lib\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.774399 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773832    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qps4w\" (UniqueName: \"kubernetes.io/projected/0557c247-c431-4eef-9de0-0b272b9e5250-kube-api-access-qps4w\") pod \"node-ca-ddfsj\" (UID: \"0557c247-c431-4eef-9de0-0b272b9e5250\") " pod="openshift-image-registry/node-ca-ddfsj"
Jul 26 17:49:05.774399 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773856    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b2473540-1734-4f1a-ab9a-719694dd5c96-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mbf4c\" (UID: \"b2473540-1734-4f1a-ab9a-719694dd5c96\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c"
Jul 26 17:49:05.774399 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773878    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-host-var-lib-kubelet\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.774399 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773901    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fcc71092-cd57-404f-aa06-8a3d8486eb33-iptables-alerter-script\") pod \"iptables-alerter-8sbv6\" (UID: \"fcc71092-cd57-404f-aa06-8a3d8486eb33\") " pod="openshift-network-operator/iptables-alerter-8sbv6"
Jul 26 17:49:05.774399 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773924    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/048c9bc9-3ff2-4667-8392-103e4e235f0a-tmp\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.774399 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773950    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-log-socket\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.774399 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.773992    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b2473540-1734-4f1a-ab9a-719694dd5c96-etc-selinux\") pod \"aws-ebs-csi-driver-node-mbf4c\" (UID: \"b2473540-1734-4f1a-ab9a-719694dd5c96\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c"
Jul 26 17:49:05.774399 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.774024    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79g49\" (UniqueName: \"kubernetes.io/projected/a337e0eb-4d13-4df5-bfe7-186de89d4dcc-kube-api-access-79g49\") pod \"multus-additional-cni-plugins-s4zzb\" (UID: \"a337e0eb-4d13-4df5-bfe7-186de89d4dcc\") " pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.774399 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.774049    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-host-var-lib-cni-bin\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.774399 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.774073    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-run-openvswitch\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.774399 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.774095    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3f795165-d9b9-4634-a24e-d10cee628e00-ovnkube-config\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.774399 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.774134    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b2473540-1734-4f1a-ab9a-719694dd5c96-sys-fs\") pod \"aws-ebs-csi-driver-node-mbf4c\" (UID: \"b2473540-1734-4f1a-ab9a-719694dd5c96\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c"
Jul 26 17:49:05.774399 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.774157    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-etc-systemd\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.774399 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.774179    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0557c247-c431-4eef-9de0-0b272b9e5250-serviceca\") pod \"node-ca-ddfsj\" (UID: \"0557c247-c431-4eef-9de0-0b272b9e5250\") " pod="openshift-image-registry/node-ca-ddfsj"
Jul 26 17:49:05.774399 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.774202    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a337e0eb-4d13-4df5-bfe7-186de89d4dcc-os-release\") pod \"multus-additional-cni-plugins-s4zzb\" (UID: \"a337e0eb-4d13-4df5-bfe7-186de89d4dcc\") " pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.775203 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.774227    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a337e0eb-4d13-4df5-bfe7-186de89d4dcc-system-cni-dir\") pod \"multus-additional-cni-plugins-s4zzb\" (UID: \"a337e0eb-4d13-4df5-bfe7-186de89d4dcc\") " pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.775203 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.774250    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-os-release\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.775203 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.774274    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/32cbd054-d7a7-4c00-be2e-012e16598a49-kubelet-config\") pod \"global-pull-secret-syncer-96jft\" (UID: \"32cbd054-d7a7-4c00-be2e-012e16598a49\") " pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:05.775203 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.774481    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32cbd054-d7a7-4c00-be2e-012e16598a49-original-pull-secret\") pod \"global-pull-secret-syncer-96jft\" (UID: \"32cbd054-d7a7-4c00-be2e-012e16598a49\") " pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:05.775203 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.774533    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-etc-openvswitch\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.775203 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.774571    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0557c247-c431-4eef-9de0-0b272b9e5250-host\") pod \"node-ca-ddfsj\" (UID: \"0557c247-c431-4eef-9de0-0b272b9e5250\") " pod="openshift-image-registry/node-ca-ddfsj"
Jul 26 17:49:05.775203 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.774602    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b2473540-1734-4f1a-ab9a-719694dd5c96-socket-dir\") pod \"aws-ebs-csi-driver-node-mbf4c\" (UID: \"b2473540-1734-4f1a-ab9a-719694dd5c96\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c"
Jul 26 17:49:05.775203 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.774740    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b2473540-1734-4f1a-ab9a-719694dd5c96-device-dir\") pod \"aws-ebs-csi-driver-node-mbf4c\" (UID: \"b2473540-1734-4f1a-ab9a-719694dd5c96\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c"
Jul 26 17:49:05.775203 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.774773    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j4q2\" (UniqueName: \"kubernetes.io/projected/af7dc810-9c37-4155-ab59-7c4196ffb50b-kube-api-access-4j4q2\") pod \"network-metrics-daemon-7mtnm\" (UID: \"af7dc810-9c37-4155-ab59-7c4196ffb50b\") " pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:05.775203 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.774805    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-host\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.775203 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.774836    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vl7qz\" (UniqueName: \"kubernetes.io/projected/3f795165-d9b9-4634-a24e-d10cee628e00-kube-api-access-vl7qz\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.775203 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.775150    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-multus-cni-dir\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.775203 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.775185    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-multus-socket-dir-parent\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.775818 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.775217    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-host-kubelet\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.775818 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.775249    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-host-cni-netd\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.775818 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.775369    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-host-var-lib-cni-multus\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.775818 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.775396    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5pjm\" (UniqueName: \"kubernetes.io/projected/63076f7b-11ca-42d6-8d7b-2e5a96035317-kube-api-access-t5pjm\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.775818 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.775427    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-etc-modprobe-d\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.775818 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.775458    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-etc-sysctl-d\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.775818 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.775486    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-sys\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.775818 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.775518    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-host-slash\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.775818 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.775655    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a337e0eb-4d13-4df5-bfe7-186de89d4dcc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s4zzb\" (UID: \"a337e0eb-4d13-4df5-bfe7-186de89d4dcc\") " pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.775818 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.775687    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-etc-sysctl-conf\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.775818 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.775717    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-node-log\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.782888 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.782862    2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 26 17:49:05.802644 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.802594    2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7bjsh"
Jul 26 17:49:05.811160 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.810895    2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7bjsh"
Jul 26 17:49:05.837594 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.837548    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-45.ec2.internal" event={"ID":"2ae3bee3a2b4c14b81fcf318d81e0f7e","Type":"ContainerStarted","Data":"efadbc58422aceeb7372aad767330fb412f5a15037596ec2f29d11ba1c7eb93b"}
Jul 26 17:49:05.839846 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.839812    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-45.ec2.internal" event={"ID":"e256c610577a2f8cee15b21c5c7fed95","Type":"ContainerStarted","Data":"00c24fd9f29b763d092dc3317b240aaf83d074f1b0637317a3a7e8a9b542b995"}
Jul 26 17:49:05.862947 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.862928    2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 26 17:49:05.876919 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.876891    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a337e0eb-4d13-4df5-bfe7-186de89d4dcc-cnibin\") pod \"multus-additional-cni-plugins-s4zzb\" (UID: \"a337e0eb-4d13-4df5-bfe7-186de89d4dcc\") " pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.877031 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.876935    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/63076f7b-11ca-42d6-8d7b-2e5a96035317-multus-daemon-config\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.877031 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.876963    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z24fd\" (UniqueName: \"kubernetes.io/projected/fcc71092-cd57-404f-aa06-8a3d8486eb33-kube-api-access-z24fd\") pod \"iptables-alerter-8sbv6\" (UID: \"fcc71092-cd57-404f-aa06-8a3d8486eb33\") " pod="openshift-network-operator/iptables-alerter-8sbv6"
Jul 26 17:49:05.877031 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.876987    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-var-lib-openvswitch\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.877031 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877010    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-host-run-ovn-kubernetes\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.877261 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877031    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a337e0eb-4d13-4df5-bfe7-186de89d4dcc-cnibin\") pod \"multus-additional-cni-plugins-s4zzb\" (UID: \"a337e0eb-4d13-4df5-bfe7-186de89d4dcc\") " pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.877261 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877050    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3f795165-d9b9-4634-a24e-d10cee628e00-env-overrides\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.877261 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877088    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3514ccab-ab9c-46b1-bf1a-21cac46c7586-agent-certs\") pod \"konnectivity-agent-khbbn\" (UID: \"3514ccab-ab9c-46b1-bf1a-21cac46c7586\") " pod="kube-system/konnectivity-agent-khbbn"
Jul 26 17:49:05.877261 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877133    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-run-systemd\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.877261 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877145    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-var-lib-openvswitch\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.877261 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877160    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-run-ovn\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.877261 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877182    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-host-cni-bin\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.877261 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877195    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-host-run-ovn-kubernetes\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.877261 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877207    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.877261 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877233    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3f795165-d9b9-4634-a24e-d10cee628e00-ovnkube-script-lib\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.877261 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877259    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qps4w\" (UniqueName: \"kubernetes.io/projected/0557c247-c431-4eef-9de0-0b272b9e5250-kube-api-access-qps4w\") pod \"node-ca-ddfsj\" (UID: \"0557c247-c431-4eef-9de0-0b272b9e5250\") " pod="openshift-image-registry/node-ca-ddfsj"
Jul 26 17:49:05.877772 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877284    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b2473540-1734-4f1a-ab9a-719694dd5c96-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mbf4c\" (UID: \"b2473540-1734-4f1a-ab9a-719694dd5c96\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c"
Jul 26 17:49:05.877772 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877305    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-host-var-lib-kubelet\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.877772 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877335    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fcc71092-cd57-404f-aa06-8a3d8486eb33-iptables-alerter-script\") pod \"iptables-alerter-8sbv6\" (UID: \"fcc71092-cd57-404f-aa06-8a3d8486eb33\") " pod="openshift-network-operator/iptables-alerter-8sbv6"
Jul 26 17:49:05.877772 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877359    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/048c9bc9-3ff2-4667-8392-103e4e235f0a-tmp\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.877772 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877383    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-log-socket\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.877772 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877408    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b2473540-1734-4f1a-ab9a-719694dd5c96-etc-selinux\") pod \"aws-ebs-csi-driver-node-mbf4c\" (UID: \"b2473540-1734-4f1a-ab9a-719694dd5c96\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c"
Jul 26 17:49:05.877772 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877432    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79g49\" (UniqueName: \"kubernetes.io/projected/a337e0eb-4d13-4df5-bfe7-186de89d4dcc-kube-api-access-79g49\") pod \"multus-additional-cni-plugins-s4zzb\" (UID: \"a337e0eb-4d13-4df5-bfe7-186de89d4dcc\") " pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.877772 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877457    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-host-var-lib-cni-bin\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.877772 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877482    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-run-openvswitch\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.877772 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877509    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b2473540-1734-4f1a-ab9a-719694dd5c96-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mbf4c\" (UID: \"b2473540-1734-4f1a-ab9a-719694dd5c96\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c"
Jul 26 17:49:05.877772 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877522    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3f795165-d9b9-4634-a24e-d10cee628e00-ovnkube-config\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.877772 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877549    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b2473540-1734-4f1a-ab9a-719694dd5c96-sys-fs\") pod \"aws-ebs-csi-driver-node-mbf4c\" (UID: \"b2473540-1734-4f1a-ab9a-719694dd5c96\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c"
Jul 26 17:49:05.877772 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877558    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-run-ovn\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.877772 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877573    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-etc-systemd\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.877772 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877590    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-host-cni-bin\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.877772 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877598    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0557c247-c431-4eef-9de0-0b272b9e5250-serviceca\") pod \"node-ca-ddfsj\" (UID: \"0557c247-c431-4eef-9de0-0b272b9e5250\") " pod="openshift-image-registry/node-ca-ddfsj"
Jul 26 17:49:05.877772 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877624    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a337e0eb-4d13-4df5-bfe7-186de89d4dcc-os-release\") pod \"multus-additional-cni-plugins-s4zzb\" (UID: \"a337e0eb-4d13-4df5-bfe7-186de89d4dcc\") " pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.878572 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877627    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.878572 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877631    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/63076f7b-11ca-42d6-8d7b-2e5a96035317-multus-daemon-config\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.878572 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877649    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3514ccab-ab9c-46b1-bf1a-21cac46c7586-konnectivity-ca\") pod \"konnectivity-agent-khbbn\" (UID: \"3514ccab-ab9c-46b1-bf1a-21cac46c7586\") " pod="kube-system/konnectivity-agent-khbbn"
Jul 26 17:49:05.878572 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877689    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a337e0eb-4d13-4df5-bfe7-186de89d4dcc-system-cni-dir\") pod \"multus-additional-cni-plugins-s4zzb\" (UID: \"a337e0eb-4d13-4df5-bfe7-186de89d4dcc\") " pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.878572 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877236    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-run-systemd\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.878572 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877712    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-os-release\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.878572 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877755    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/32cbd054-d7a7-4c00-be2e-012e16598a49-kubelet-config\") pod \"global-pull-secret-syncer-96jft\" (UID: \"32cbd054-d7a7-4c00-be2e-012e16598a49\") " pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:05.878572 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877781    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-os-release\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.878572 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877784    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32cbd054-d7a7-4c00-be2e-012e16598a49-original-pull-secret\") pod \"global-pull-secret-syncer-96jft\" (UID: \"32cbd054-d7a7-4c00-be2e-012e16598a49\") " pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:05.878572 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877826    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-host-var-lib-kubelet\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.878572 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:05.877861    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 17:49:05.878572 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:05.877927    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32cbd054-d7a7-4c00-be2e-012e16598a49-original-pull-secret podName:32cbd054-d7a7-4c00-be2e-012e16598a49 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:06.37790836 +0000 UTC m=+3.030099213 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/32cbd054-d7a7-4c00-be2e-012e16598a49-original-pull-secret") pod "global-pull-secret-syncer-96jft" (UID: "32cbd054-d7a7-4c00-be2e-012e16598a49") : object "kube-system"/"original-pull-secret" not registered
Jul 26 17:49:05.878572 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877922    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/32cbd054-d7a7-4c00-be2e-012e16598a49-kubelet-config\") pod \"global-pull-secret-syncer-96jft\" (UID: \"32cbd054-d7a7-4c00-be2e-012e16598a49\") " pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:05.878572 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877949    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-etc-openvswitch\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.878572 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.877979    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0557c247-c431-4eef-9de0-0b272b9e5250-host\") pod \"node-ca-ddfsj\" (UID: \"0557c247-c431-4eef-9de0-0b272b9e5250\") " pod="openshift-image-registry/node-ca-ddfsj"
Jul 26 17:49:05.878572 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878011    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b2473540-1734-4f1a-ab9a-719694dd5c96-socket-dir\") pod \"aws-ebs-csi-driver-node-mbf4c\" (UID: \"b2473540-1734-4f1a-ab9a-719694dd5c96\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c"
Jul 26 17:49:05.878572 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878037    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b2473540-1734-4f1a-ab9a-719694dd5c96-device-dir\") pod \"aws-ebs-csi-driver-node-mbf4c\" (UID: \"b2473540-1734-4f1a-ab9a-719694dd5c96\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c"
Jul 26 17:49:05.879360 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878060    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4j4q2\" (UniqueName: \"kubernetes.io/projected/af7dc810-9c37-4155-ab59-7c4196ffb50b-kube-api-access-4j4q2\") pod \"network-metrics-daemon-7mtnm\" (UID: \"af7dc810-9c37-4155-ab59-7c4196ffb50b\") " pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:05.879360 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878083    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-host\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.879360 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878108    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vl7qz\" (UniqueName: \"kubernetes.io/projected/3f795165-d9b9-4634-a24e-d10cee628e00-kube-api-access-vl7qz\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.879360 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878149    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-multus-cni-dir\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.879360 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878177    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-multus-socket-dir-parent\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.879360 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878202    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-host-kubelet\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.879360 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878200    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b2473540-1734-4f1a-ab9a-719694dd5c96-etc-selinux\") pod \"aws-ebs-csi-driver-node-mbf4c\" (UID: \"b2473540-1734-4f1a-ab9a-719694dd5c96\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c"
Jul 26 17:49:05.879360 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878251    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-etc-openvswitch\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.879360 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878251    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-host-kubelet\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.879360 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878279    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3f795165-d9b9-4634-a24e-d10cee628e00-ovnkube-script-lib\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.879360 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878301    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0557c247-c431-4eef-9de0-0b272b9e5250-host\") pod \"node-ca-ddfsj\" (UID: \"0557c247-c431-4eef-9de0-0b272b9e5250\") " pod="openshift-image-registry/node-ca-ddfsj"
Jul 26 17:49:05.879360 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878324    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-host-var-lib-cni-bin\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.879360 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878380    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fcc71092-cd57-404f-aa06-8a3d8486eb33-iptables-alerter-script\") pod \"iptables-alerter-8sbv6\" (UID: \"fcc71092-cd57-404f-aa06-8a3d8486eb33\") " pod="openshift-network-operator/iptables-alerter-8sbv6"
Jul 26 17:49:05.879360 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878384    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b2473540-1734-4f1a-ab9a-719694dd5c96-socket-dir\") pod \"aws-ebs-csi-driver-node-mbf4c\" (UID: \"b2473540-1734-4f1a-ab9a-719694dd5c96\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c"
Jul 26 17:49:05.879360 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878410    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b2473540-1734-4f1a-ab9a-719694dd5c96-device-dir\") pod \"aws-ebs-csi-driver-node-mbf4c\" (UID: \"b2473540-1734-4f1a-ab9a-719694dd5c96\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c"
Jul 26 17:49:05.879360 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878452    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-host\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.879360 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878497    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b2473540-1734-4f1a-ab9a-719694dd5c96-sys-fs\") pod \"aws-ebs-csi-driver-node-mbf4c\" (UID: \"b2473540-1734-4f1a-ab9a-719694dd5c96\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c"
Jul 26 17:49:05.879360 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878499    2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 26 17:49:05.880248 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878521    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-etc-systemd\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.880248 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878560    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a337e0eb-4d13-4df5-bfe7-186de89d4dcc-os-release\") pod \"multus-additional-cni-plugins-s4zzb\" (UID: \"a337e0eb-4d13-4df5-bfe7-186de89d4dcc\") " pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.880248 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878669    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-log-socket\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.880248 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878713    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a337e0eb-4d13-4df5-bfe7-186de89d4dcc-system-cni-dir\") pod \"multus-additional-cni-plugins-s4zzb\" (UID: \"a337e0eb-4d13-4df5-bfe7-186de89d4dcc\") " pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.880248 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878750    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-host-cni-netd\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.880248 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878769    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-run-openvswitch\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.880248 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878776    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-multus-cni-dir\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.880248 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878818    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-host-cni-netd\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.880248 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878843    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-multus-socket-dir-parent\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.880248 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878853    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-host-var-lib-cni-multus\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.880248 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878890    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-host-var-lib-cni-multus\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.880248 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878891    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5pjm\" (UniqueName: \"kubernetes.io/projected/63076f7b-11ca-42d6-8d7b-2e5a96035317-kube-api-access-t5pjm\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.880248 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878925    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-etc-modprobe-d\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.880248 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.878953    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0557c247-c431-4eef-9de0-0b272b9e5250-serviceca\") pod \"node-ca-ddfsj\" (UID: \"0557c247-c431-4eef-9de0-0b272b9e5250\") " pod="openshift-image-registry/node-ca-ddfsj"
Jul 26 17:49:05.880248 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879027    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3f795165-d9b9-4634-a24e-d10cee628e00-env-overrides\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.880248 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879035    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-etc-sysctl-d\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.880248 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879065    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-sys\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.880248 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879102    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-host-slash\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.881099 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879104    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-etc-modprobe-d\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.881099 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879143    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a337e0eb-4d13-4df5-bfe7-186de89d4dcc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s4zzb\" (UID: \"a337e0eb-4d13-4df5-bfe7-186de89d4dcc\") " pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.881099 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879154    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-etc-sysctl-d\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.881099 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879170    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-etc-sysctl-conf\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.881099 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879178    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-sys\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.881099 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879190    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-host-slash\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.881099 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879194    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-node-log\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.881099 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879222    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jntqg\" (UniqueName: \"kubernetes.io/projected/76a782a1-4344-459a-8f5c-3e84c018f0f7-kube-api-access-jntqg\") pod \"network-check-target-h8sjb\" (UID: \"76a782a1-4344-459a-8f5c-3e84c018f0f7\") " pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:05.881099 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879246    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gwl4\" (UniqueName: \"kubernetes.io/projected/b2473540-1734-4f1a-ab9a-719694dd5c96-kube-api-access-8gwl4\") pod \"aws-ebs-csi-driver-node-mbf4c\" (UID: \"b2473540-1734-4f1a-ab9a-719694dd5c96\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c"
Jul 26 17:49:05.881099 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879275    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a337e0eb-4d13-4df5-bfe7-186de89d4dcc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s4zzb\" (UID: \"a337e0eb-4d13-4df5-bfe7-186de89d4dcc\") " pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.881099 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879293    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-etc-sysctl-conf\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.881099 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879301    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/63076f7b-11ca-42d6-8d7b-2e5a96035317-cni-binary-copy\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.881099 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879321    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3f795165-d9b9-4634-a24e-d10cee628e00-ovnkube-config\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.881099 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879343    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-host-run-multus-certs\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.881099 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879368    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-etc-kubernetes\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.881099 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879390    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fcc71092-cd57-404f-aa06-8a3d8486eb33-host-slash\") pod \"iptables-alerter-8sbv6\" (UID: \"fcc71092-cd57-404f-aa06-8a3d8486eb33\") " pod="openshift-network-operator/iptables-alerter-8sbv6"
Jul 26 17:49:05.881099 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879413    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmzxt\" (UniqueName: \"kubernetes.io/projected/1fcdab4c-350c-44c3-9bc0-a9995d33c539-kube-api-access-dmzxt\") pod \"node-resolver-974ql\" (UID: \"1fcdab4c-350c-44c3-9bc0-a9995d33c539\") " pod="openshift-dns/node-resolver-974ql"
Jul 26 17:49:05.881990 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879436    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-hostroot\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.881990 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879457    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-lib-modules\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.881990 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879489    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/32cbd054-d7a7-4c00-be2e-012e16598a49-dbus\") pod \"global-pull-secret-syncer-96jft\" (UID: \"32cbd054-d7a7-4c00-be2e-012e16598a49\") " pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:05.881990 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879511    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3f795165-d9b9-4634-a24e-d10cee628e00-ovn-node-metrics-cert\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.881990 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879534    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a337e0eb-4d13-4df5-bfe7-186de89d4dcc-cni-binary-copy\") pod \"multus-additional-cni-plugins-s4zzb\" (UID: \"a337e0eb-4d13-4df5-bfe7-186de89d4dcc\") " pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.881990 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879552    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-cnibin\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.881990 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879570    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-run\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.881990 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879589    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/048c9bc9-3ff2-4667-8392-103e4e235f0a-etc-tuned\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.881990 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879609    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fcdab4c-350c-44c3-9bc0-a9995d33c539-tmp-dir\") pod \"node-resolver-974ql\" (UID: \"1fcdab4c-350c-44c3-9bc0-a9995d33c539\") " pod="openshift-dns/node-resolver-974ql"
Jul 26 17:49:05.881990 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879629    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b2473540-1734-4f1a-ab9a-719694dd5c96-registration-dir\") pod \"aws-ebs-csi-driver-node-mbf4c\" (UID: \"b2473540-1734-4f1a-ab9a-719694dd5c96\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c"
Jul 26 17:49:05.881990 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879648    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-host-run-k8s-cni-cncf-io\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.881990 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879668    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-host-run-netns\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.881990 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879688    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af7dc810-9c37-4155-ab59-7c4196ffb50b-metrics-certs\") pod \"network-metrics-daemon-7mtnm\" (UID: \"af7dc810-9c37-4155-ab59-7c4196ffb50b\") " pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:05.881990 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879708    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-etc-sysconfig\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.881990 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879711    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a337e0eb-4d13-4df5-bfe7-186de89d4dcc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s4zzb\" (UID: \"a337e0eb-4d13-4df5-bfe7-186de89d4dcc\") " pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.881990 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879738    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-node-log\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.881990 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879760    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-etc-sysconfig\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.881990 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879771    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-host-run-multus-certs\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.882837 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879823    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a337e0eb-4d13-4df5-bfe7-186de89d4dcc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s4zzb\" (UID: \"a337e0eb-4d13-4df5-bfe7-186de89d4dcc\") " pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.882837 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879712    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/63076f7b-11ca-42d6-8d7b-2e5a96035317-cni-binary-copy\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.882837 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879852    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-etc-kubernetes\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.882837 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879813    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-etc-kubernetes\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.882837 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879859    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fcc71092-cd57-404f-aa06-8a3d8486eb33-host-slash\") pod \"iptables-alerter-8sbv6\" (UID: \"fcc71092-cd57-404f-aa06-8a3d8486eb33\") " pod="openshift-network-operator/iptables-alerter-8sbv6"
Jul 26 17:49:05.882837 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879879    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1fcdab4c-350c-44c3-9bc0-a9995d33c539-hosts-file\") pod \"node-resolver-974ql\" (UID: \"1fcdab4c-350c-44c3-9bc0-a9995d33c539\") " pod="openshift-dns/node-resolver-974ql"
Jul 26 17:49:05.882837 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879897    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a337e0eb-4d13-4df5-bfe7-186de89d4dcc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s4zzb\" (UID: \"a337e0eb-4d13-4df5-bfe7-186de89d4dcc\") " pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.882837 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879902    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-etc-kubernetes\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.882837 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879912    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-system-cni-dir\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.882837 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879944    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-system-cni-dir\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.882837 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879972    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1fcdab4c-350c-44c3-9bc0-a9995d33c539-hosts-file\") pod \"node-resolver-974ql\" (UID: \"1fcdab4c-350c-44c3-9bc0-a9995d33c539\") " pod="openshift-dns/node-resolver-974ql"
Jul 26 17:49:05.882837 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.879977    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-lib-modules\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.882837 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.880057    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a337e0eb-4d13-4df5-bfe7-186de89d4dcc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s4zzb\" (UID: \"a337e0eb-4d13-4df5-bfe7-186de89d4dcc\") " pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.882837 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.880079    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-hostroot\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.882837 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.880133    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/32cbd054-d7a7-4c00-be2e-012e16598a49-dbus\") pod \"global-pull-secret-syncer-96jft\" (UID: \"32cbd054-d7a7-4c00-be2e-012e16598a49\") " pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:05.882837 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.880180    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-host-run-netns\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.882837 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.880182    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-cnibin\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.882837 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.880214    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b2473540-1734-4f1a-ab9a-719694dd5c96-registration-dir\") pod \"aws-ebs-csi-driver-node-mbf4c\" (UID: \"b2473540-1734-4f1a-ab9a-719694dd5c96\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c"
Jul 26 17:49:05.883775 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.880249    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fcdab4c-350c-44c3-9bc0-a9995d33c539-tmp-dir\") pod \"node-resolver-974ql\" (UID: \"1fcdab4c-350c-44c3-9bc0-a9995d33c539\") " pod="openshift-dns/node-resolver-974ql"
Jul 26 17:49:05.883775 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.880255    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-host-run-k8s-cni-cncf-io\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.883775 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:05.880268    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:49:05.883775 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.880304    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-run\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.883775 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:05.880330    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af7dc810-9c37-4155-ab59-7c4196ffb50b-metrics-certs podName:af7dc810-9c37-4155-ab59-7c4196ffb50b nodeName:}" failed. No retries permitted until 2026-07-26 17:49:06.380312691 +0000 UTC m=+3.032503542 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af7dc810-9c37-4155-ab59-7c4196ffb50b-metrics-certs") pod "network-metrics-daemon-7mtnm" (UID: "af7dc810-9c37-4155-ab59-7c4196ffb50b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:49:05.883775 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.880408    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-multus-conf-dir\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.883775 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.880438    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a337e0eb-4d13-4df5-bfe7-186de89d4dcc-cni-binary-copy\") pod \"multus-additional-cni-plugins-s4zzb\" (UID: \"a337e0eb-4d13-4df5-bfe7-186de89d4dcc\") " pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.883775 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.880443    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-var-lib-kubelet\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.883775 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.880489    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/63076f7b-11ca-42d6-8d7b-2e5a96035317-multus-conf-dir\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.883775 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.880493    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6v2h\" (UniqueName: \"kubernetes.io/projected/048c9bc9-3ff2-4667-8392-103e4e235f0a-kube-api-access-z6v2h\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.883775 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.880529    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-systemd-units\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.883775 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.880490    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/048c9bc9-3ff2-4667-8392-103e4e235f0a-var-lib-kubelet\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.883775 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.880561    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-host-run-netns\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.883775 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.880576    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-systemd-units\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.883775 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.880596    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f795165-d9b9-4634-a24e-d10cee628e00-host-run-netns\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.883775 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.883424    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/048c9bc9-3ff2-4667-8392-103e4e235f0a-tmp\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.883775 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.883461    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/048c9bc9-3ff2-4667-8392-103e4e235f0a-etc-tuned\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.885584 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.885563    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3f795165-d9b9-4634-a24e-d10cee628e00-ovn-node-metrics-cert\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.887059 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.887029    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79g49\" (UniqueName: \"kubernetes.io/projected/a337e0eb-4d13-4df5-bfe7-186de89d4dcc-kube-api-access-79g49\") pod \"multus-additional-cni-plugins-s4zzb\" (UID: \"a337e0eb-4d13-4df5-bfe7-186de89d4dcc\") " pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:05.887415 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.887392    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z24fd\" (UniqueName: \"kubernetes.io/projected/fcc71092-cd57-404f-aa06-8a3d8486eb33-kube-api-access-z24fd\") pod \"iptables-alerter-8sbv6\" (UID: \"fcc71092-cd57-404f-aa06-8a3d8486eb33\") " pod="openshift-network-operator/iptables-alerter-8sbv6"
Jul 26 17:49:05.887905 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:05.887887    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 17:49:05.887975 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:05.887911    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 17:49:05.887975 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:05.887924    2570 projected.go:196] Error preparing data for projected volume kube-api-access-jntqg for pod openshift-network-diagnostics/network-check-target-h8sjb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:49:05.888086 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:05.887992    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76a782a1-4344-459a-8f5c-3e84c018f0f7-kube-api-access-jntqg podName:76a782a1-4344-459a-8f5c-3e84c018f0f7 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:06.387977192 +0000 UTC m=+3.040168036 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jntqg" (UniqueName: "kubernetes.io/projected/76a782a1-4344-459a-8f5c-3e84c018f0f7-kube-api-access-jntqg") pod "network-check-target-h8sjb" (UID: "76a782a1-4344-459a-8f5c-3e84c018f0f7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:49:05.890611 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.889834    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5pjm\" (UniqueName: \"kubernetes.io/projected/63076f7b-11ca-42d6-8d7b-2e5a96035317-kube-api-access-t5pjm\") pod \"multus-tslqc\" (UID: \"63076f7b-11ca-42d6-8d7b-2e5a96035317\") " pod="openshift-multus/multus-tslqc"
Jul 26 17:49:05.890611 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.890071    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gwl4\" (UniqueName: \"kubernetes.io/projected/b2473540-1734-4f1a-ab9a-719694dd5c96-kube-api-access-8gwl4\") pod \"aws-ebs-csi-driver-node-mbf4c\" (UID: \"b2473540-1734-4f1a-ab9a-719694dd5c96\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c"
Jul 26 17:49:05.890611 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.890480    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qps4w\" (UniqueName: \"kubernetes.io/projected/0557c247-c431-4eef-9de0-0b272b9e5250-kube-api-access-qps4w\") pod \"node-ca-ddfsj\" (UID: \"0557c247-c431-4eef-9de0-0b272b9e5250\") " pod="openshift-image-registry/node-ca-ddfsj"
Jul 26 17:49:05.890886 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.890859    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl7qz\" (UniqueName: \"kubernetes.io/projected/3f795165-d9b9-4634-a24e-d10cee628e00-kube-api-access-vl7qz\") pod \"ovnkube-node-5zh25\" (UID: \"3f795165-d9b9-4634-a24e-d10cee628e00\") " pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:05.892041 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.891902    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmzxt\" (UniqueName: \"kubernetes.io/projected/1fcdab4c-350c-44c3-9bc0-a9995d33c539-kube-api-access-dmzxt\") pod \"node-resolver-974ql\" (UID: \"1fcdab4c-350c-44c3-9bc0-a9995d33c539\") " pod="openshift-dns/node-resolver-974ql"
Jul 26 17:49:05.892957 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.892933    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j4q2\" (UniqueName: \"kubernetes.io/projected/af7dc810-9c37-4155-ab59-7c4196ffb50b-kube-api-access-4j4q2\") pod \"network-metrics-daemon-7mtnm\" (UID: \"af7dc810-9c37-4155-ab59-7c4196ffb50b\") " pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:05.893049 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.892965    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6v2h\" (UniqueName: \"kubernetes.io/projected/048c9bc9-3ff2-4667-8392-103e4e235f0a-kube-api-access-z6v2h\") pod \"tuned-ccd9g\" (UID: \"048c9bc9-3ff2-4667-8392-103e4e235f0a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:05.981435 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.981354    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3514ccab-ab9c-46b1-bf1a-21cac46c7586-agent-certs\") pod \"konnectivity-agent-khbbn\" (UID: \"3514ccab-ab9c-46b1-bf1a-21cac46c7586\") " pod="kube-system/konnectivity-agent-khbbn"
Jul 26 17:49:05.981435 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.981395    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3514ccab-ab9c-46b1-bf1a-21cac46c7586-konnectivity-ca\") pod \"konnectivity-agent-khbbn\" (UID: \"3514ccab-ab9c-46b1-bf1a-21cac46c7586\") " pod="kube-system/konnectivity-agent-khbbn"
Jul 26 17:49:05.981869 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.981845    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3514ccab-ab9c-46b1-bf1a-21cac46c7586-konnectivity-ca\") pod \"konnectivity-agent-khbbn\" (UID: \"3514ccab-ab9c-46b1-bf1a-21cac46c7586\") " pod="kube-system/konnectivity-agent-khbbn"
Jul 26 17:49:05.983995 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:05.983961    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3514ccab-ab9c-46b1-bf1a-21cac46c7586-agent-certs\") pod \"konnectivity-agent-khbbn\" (UID: \"3514ccab-ab9c-46b1-bf1a-21cac46c7586\") " pod="kube-system/konnectivity-agent-khbbn"
Jul 26 17:49:06.071269 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:06.071226    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s4zzb"
Jul 26 17:49:06.089166 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:06.089137    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tslqc"
Jul 26 17:49:06.099746 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:06.099725    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8sbv6"
Jul 26 17:49:06.106472 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:06.106446    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:06.115190 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:06.115168    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-974ql"
Jul 26 17:49:06.121763 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:06.121746    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ddfsj"
Jul 26 17:49:06.128283 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:06.128258    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c"
Jul 26 17:49:06.136894 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:06.136877    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ccd9g"
Jul 26 17:49:06.143263 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:06.143247    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-khbbn"
Jul 26 17:49:06.384004 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:06.383959    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32cbd054-d7a7-4c00-be2e-012e16598a49-original-pull-secret\") pod \"global-pull-secret-syncer-96jft\" (UID: \"32cbd054-d7a7-4c00-be2e-012e16598a49\") " pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:06.384221 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:06.384034    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af7dc810-9c37-4155-ab59-7c4196ffb50b-metrics-certs\") pod \"network-metrics-daemon-7mtnm\" (UID: \"af7dc810-9c37-4155-ab59-7c4196ffb50b\") " pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:06.384221 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:06.384140    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 17:49:06.384221 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:06.384216    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32cbd054-d7a7-4c00-be2e-012e16598a49-original-pull-secret podName:32cbd054-d7a7-4c00-be2e-012e16598a49 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:07.384201042 +0000 UTC m=+4.036391878 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/32cbd054-d7a7-4c00-be2e-012e16598a49-original-pull-secret") pod "global-pull-secret-syncer-96jft" (UID: "32cbd054-d7a7-4c00-be2e-012e16598a49") : object "kube-system"/"original-pull-secret" not registered
Jul 26 17:49:06.384393 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:06.384148    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:49:06.384393 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:06.384299    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af7dc810-9c37-4155-ab59-7c4196ffb50b-metrics-certs podName:af7dc810-9c37-4155-ab59-7c4196ffb50b nodeName:}" failed. No retries permitted until 2026-07-26 17:49:07.384281269 +0000 UTC m=+4.036472121 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af7dc810-9c37-4155-ab59-7c4196ffb50b-metrics-certs") pod "network-metrics-daemon-7mtnm" (UID: "af7dc810-9c37-4155-ab59-7c4196ffb50b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:49:06.485328 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:06.485285    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jntqg\" (UniqueName: \"kubernetes.io/projected/76a782a1-4344-459a-8f5c-3e84c018f0f7-kube-api-access-jntqg\") pod \"network-check-target-h8sjb\" (UID: \"76a782a1-4344-459a-8f5c-3e84c018f0f7\") " pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:06.485528 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:06.485475    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 17:49:06.485528 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:06.485506    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 17:49:06.485528 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:06.485518    2570 projected.go:196] Error preparing data for projected volume kube-api-access-jntqg for pod openshift-network-diagnostics/network-check-target-h8sjb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:49:06.485673 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:06.485584    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76a782a1-4344-459a-8f5c-3e84c018f0f7-kube-api-access-jntqg podName:76a782a1-4344-459a-8f5c-3e84c018f0f7 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:07.485565772 +0000 UTC m=+4.137756627 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jntqg" (UniqueName: "kubernetes.io/projected/76a782a1-4344-459a-8f5c-3e84c018f0f7-kube-api-access-jntqg") pod "network-check-target-h8sjb" (UID: "76a782a1-4344-459a-8f5c-3e84c018f0f7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:49:06.816283 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:06.816242    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-25 17:44:05 +0000 UTC" deadline="2028-04-09 07:02:43.725937248 +0000 UTC"
Jul 26 17:49:06.816283 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:06.816279    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14941h13m36.909663762s"
Jul 26 17:49:06.935153 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:06.935099    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda337e0eb_4d13_4df5_bfe7_186de89d4dcc.slice/crio-5667b30405f42079e65a5cf8d258e70e542aae1b93dc7d7150b361c9fdee5810 WatchSource:0}: Error finding container 5667b30405f42079e65a5cf8d258e70e542aae1b93dc7d7150b361c9fdee5810: Status 404 returned error can't find the container with id 5667b30405f42079e65a5cf8d258e70e542aae1b93dc7d7150b361c9fdee5810
Jul 26 17:49:06.950966 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:06.950942    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f795165_d9b9_4634_a24e_d10cee628e00.slice/crio-cfd6579937fb65e3ac3dc453a137e00cd404103fb26f6ec8f2f1231902787c3e WatchSource:0}: Error finding container cfd6579937fb65e3ac3dc453a137e00cd404103fb26f6ec8f2f1231902787c3e: Status 404 returned error can't find the container with id cfd6579937fb65e3ac3dc453a137e00cd404103fb26f6ec8f2f1231902787c3e
Jul 26 17:49:06.952015 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:06.951991    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3514ccab_ab9c_46b1_bf1a_21cac46c7586.slice/crio-2cebcc94b2b06bb0f8c6b4de514fc8aec365b3315fad93853c61edfca54fa96c WatchSource:0}: Error finding container 2cebcc94b2b06bb0f8c6b4de514fc8aec365b3315fad93853c61edfca54fa96c: Status 404 returned error can't find the container with id 2cebcc94b2b06bb0f8c6b4de514fc8aec365b3315fad93853c61edfca54fa96c
Jul 26 17:49:06.953283 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:06.953188    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2473540_1734_4f1a_ab9a_719694dd5c96.slice/crio-497472cf22caa5a94f5169e6b4e4644f819fc26ec6d20e8f5af1bf9a15825e65 WatchSource:0}: Error finding container 497472cf22caa5a94f5169e6b4e4644f819fc26ec6d20e8f5af1bf9a15825e65: Status 404 returned error can't find the container with id 497472cf22caa5a94f5169e6b4e4644f819fc26ec6d20e8f5af1bf9a15825e65
Jul 26 17:49:06.954048 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:06.954021    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0557c247_c431_4eef_9de0_0b272b9e5250.slice/crio-472f577a8e490dfb64e9b36ce9669199c79d5360803339b9d9130ec5d82087e6 WatchSource:0}: Error finding container 472f577a8e490dfb64e9b36ce9669199c79d5360803339b9d9130ec5d82087e6: Status 404 returned error can't find the container with id 472f577a8e490dfb64e9b36ce9669199c79d5360803339b9d9130ec5d82087e6
Jul 26 17:49:06.955265 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:06.955062    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcc71092_cd57_404f_aa06_8a3d8486eb33.slice/crio-9f5a92eb4a03809265bbdbb166bd7515f50946a332a57f070661477cd6f7227c WatchSource:0}: Error finding container 9f5a92eb4a03809265bbdbb166bd7515f50946a332a57f070661477cd6f7227c: Status 404 returned error can't find the container with id 9f5a92eb4a03809265bbdbb166bd7515f50946a332a57f070661477cd6f7227c
Jul 26 17:49:06.956049 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:06.956025    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod048c9bc9_3ff2_4667_8392_103e4e235f0a.slice/crio-12d731bba91755db6f06391867e41a4d41fb4e5298e5a3c50cd68562c8a301fb WatchSource:0}: Error finding container 12d731bba91755db6f06391867e41a4d41fb4e5298e5a3c50cd68562c8a301fb: Status 404 returned error can't find the container with id 12d731bba91755db6f06391867e41a4d41fb4e5298e5a3c50cd68562c8a301fb
Jul 26 17:49:06.958579 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:06.958559    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63076f7b_11ca_42d6_8d7b_2e5a96035317.slice/crio-424e2306baf0e2a72d66e50fe7c7e155566f96578fb0cc7cee1864f8d115dcc2 WatchSource:0}: Error finding container 424e2306baf0e2a72d66e50fe7c7e155566f96578fb0cc7cee1864f8d115dcc2: Status 404 returned error can't find the container with id 424e2306baf0e2a72d66e50fe7c7e155566f96578fb0cc7cee1864f8d115dcc2
Jul 26 17:49:06.958839 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:06.958808    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fcdab4c_350c_44c3_9bc0_a9995d33c539.slice/crio-f01d9b8bbe8b73bb98c8fa4d741ae2371cb456c519890614332749d4acea9e58 WatchSource:0}: Error finding container f01d9b8bbe8b73bb98c8fa4d741ae2371cb456c519890614332749d4acea9e58: Status 404 returned error can't find the container with id f01d9b8bbe8b73bb98c8fa4d741ae2371cb456c519890614332749d4acea9e58
Jul 26 17:49:07.395437 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:07.394882    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32cbd054-d7a7-4c00-be2e-012e16598a49-original-pull-secret\") pod \"global-pull-secret-syncer-96jft\" (UID: \"32cbd054-d7a7-4c00-be2e-012e16598a49\") " pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:07.395437 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:07.395213    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 17:49:07.395437 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:07.395227    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af7dc810-9c37-4155-ab59-7c4196ffb50b-metrics-certs\") pod \"network-metrics-daemon-7mtnm\" (UID: \"af7dc810-9c37-4155-ab59-7c4196ffb50b\") " pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:07.395437 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:07.395285    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32cbd054-d7a7-4c00-be2e-012e16598a49-original-pull-secret podName:32cbd054-d7a7-4c00-be2e-012e16598a49 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:09.39526559 +0000 UTC m=+6.047456446 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/32cbd054-d7a7-4c00-be2e-012e16598a49-original-pull-secret") pod "global-pull-secret-syncer-96jft" (UID: "32cbd054-d7a7-4c00-be2e-012e16598a49") : object "kube-system"/"original-pull-secret" not registered
Jul 26 17:49:07.395437 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:07.395346    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:49:07.395437 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:07.395398    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af7dc810-9c37-4155-ab59-7c4196ffb50b-metrics-certs podName:af7dc810-9c37-4155-ab59-7c4196ffb50b nodeName:}" failed. No retries permitted until 2026-07-26 17:49:09.395381113 +0000 UTC m=+6.047571953 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af7dc810-9c37-4155-ab59-7c4196ffb50b-metrics-certs") pod "network-metrics-daemon-7mtnm" (UID: "af7dc810-9c37-4155-ab59-7c4196ffb50b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:49:07.497003 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:07.496345    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jntqg\" (UniqueName: \"kubernetes.io/projected/76a782a1-4344-459a-8f5c-3e84c018f0f7-kube-api-access-jntqg\") pod \"network-check-target-h8sjb\" (UID: \"76a782a1-4344-459a-8f5c-3e84c018f0f7\") " pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:07.497003 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:07.496505    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 17:49:07.497003 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:07.496523    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 17:49:07.497003 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:07.496537    2570 projected.go:196] Error preparing data for projected volume kube-api-access-jntqg for pod openshift-network-diagnostics/network-check-target-h8sjb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:49:07.497003 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:07.496598    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76a782a1-4344-459a-8f5c-3e84c018f0f7-kube-api-access-jntqg podName:76a782a1-4344-459a-8f5c-3e84c018f0f7 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:09.496579552 +0000 UTC m=+6.148770396 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jntqg" (UniqueName: "kubernetes.io/projected/76a782a1-4344-459a-8f5c-3e84c018f0f7-kube-api-access-jntqg") pod "network-check-target-h8sjb" (UID: "76a782a1-4344-459a-8f5c-3e84c018f0f7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:49:07.817529 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:07.817449    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-25 17:44:05 +0000 UTC" deadline="2028-02-12 18:16:49.016003807 +0000 UTC"
Jul 26 17:49:07.818142 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:07.817553    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13584h27m41.198480885s"
Jul 26 17:49:07.834349 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:07.834317    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:07.834507 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:07.834450    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96jft" podUID="32cbd054-d7a7-4c00-be2e-012e16598a49"
Jul 26 17:49:07.834863 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:07.834836    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:07.834976 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:07.834923    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h8sjb" podUID="76a782a1-4344-459a-8f5c-3e84c018f0f7"
Jul 26 17:49:07.835040 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:07.834982    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:07.835087 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:07.835064    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtnm" podUID="af7dc810-9c37-4155-ab59-7c4196ffb50b"
Jul 26 17:49:07.853984 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:07.853934    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5zh25" event={"ID":"3f795165-d9b9-4634-a24e-d10cee628e00","Type":"ContainerStarted","Data":"cfd6579937fb65e3ac3dc453a137e00cd404103fb26f6ec8f2f1231902787c3e"}
Jul 26 17:49:07.872458 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:07.872327    2570 generic.go:356] "Generic (PLEG): container finished" podID="2ae3bee3a2b4c14b81fcf318d81e0f7e" containerID="697f553023102406f300c212a9975277b078c3156bcbe5f9bd5a26ee7a6ee24c" exitCode=0
Jul 26 17:49:07.872458 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:07.872412    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-45.ec2.internal" event={"ID":"2ae3bee3a2b4c14b81fcf318d81e0f7e","Type":"ContainerDied","Data":"697f553023102406f300c212a9975277b078c3156bcbe5f9bd5a26ee7a6ee24c"}
Jul 26 17:49:07.898691 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:07.898635    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-45.ec2.internal" event={"ID":"e256c610577a2f8cee15b21c5c7fed95","Type":"ContainerStarted","Data":"f4af1774f74874848a0c907ecf926334dd1ff40dbfebefa1afca925eaf724085"}
Jul 26 17:49:07.915887 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:07.915855    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tslqc" event={"ID":"63076f7b-11ca-42d6-8d7b-2e5a96035317","Type":"ContainerStarted","Data":"424e2306baf0e2a72d66e50fe7c7e155566f96578fb0cc7cee1864f8d115dcc2"}
Jul 26 17:49:07.930856 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:07.930779    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8sbv6" event={"ID":"fcc71092-cd57-404f-aa06-8a3d8486eb33","Type":"ContainerStarted","Data":"9f5a92eb4a03809265bbdbb166bd7515f50946a332a57f070661477cd6f7227c"}
Jul 26 17:49:07.933319 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:07.933233    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c" event={"ID":"b2473540-1734-4f1a-ab9a-719694dd5c96","Type":"ContainerStarted","Data":"497472cf22caa5a94f5169e6b4e4644f819fc26ec6d20e8f5af1bf9a15825e65"}
Jul 26 17:49:07.935299 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:07.935227    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-khbbn" event={"ID":"3514ccab-ab9c-46b1-bf1a-21cac46c7586","Type":"ContainerStarted","Data":"2cebcc94b2b06bb0f8c6b4de514fc8aec365b3315fad93853c61edfca54fa96c"}
Jul 26 17:49:07.937241 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:07.937167    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4zzb" event={"ID":"a337e0eb-4d13-4df5-bfe7-186de89d4dcc","Type":"ContainerStarted","Data":"5667b30405f42079e65a5cf8d258e70e542aae1b93dc7d7150b361c9fdee5810"}
Jul 26 17:49:07.938657 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:07.938587    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-974ql" event={"ID":"1fcdab4c-350c-44c3-9bc0-a9995d33c539","Type":"ContainerStarted","Data":"f01d9b8bbe8b73bb98c8fa4d741ae2371cb456c519890614332749d4acea9e58"}
Jul 26 17:49:07.943375 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:07.943322    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ccd9g" event={"ID":"048c9bc9-3ff2-4667-8392-103e4e235f0a","Type":"ContainerStarted","Data":"12d731bba91755db6f06391867e41a4d41fb4e5298e5a3c50cd68562c8a301fb"}
Jul 26 17:49:07.952209 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:07.952181    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ddfsj" event={"ID":"0557c247-c431-4eef-9de0-0b272b9e5250","Type":"ContainerStarted","Data":"472f577a8e490dfb64e9b36ce9669199c79d5360803339b9d9130ec5d82087e6"}
Jul 26 17:49:08.970983 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:08.970902    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-45.ec2.internal" event={"ID":"2ae3bee3a2b4c14b81fcf318d81e0f7e","Type":"ContainerStarted","Data":"67749401839dc952b5bf5a977c1d70b1dbec0148174da99b017f9287a316e920"}
Jul 26 17:49:08.987141 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:08.986742    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-45.ec2.internal" podStartSLOduration=3.9867194489999997 podStartE2EDuration="3.986719449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:49:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:07.913249727 +0000 UTC m=+4.565440586" watchObservedRunningTime="2026-07-26 17:49:08.986719449 +0000 UTC m=+5.638910311"
Jul 26 17:49:08.987141 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:08.987073    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-45.ec2.internal" podStartSLOduration=3.98705739 podStartE2EDuration="3.98705739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:49:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:08.986205724 +0000 UTC m=+5.638396588" watchObservedRunningTime="2026-07-26 17:49:08.98705739 +0000 UTC m=+5.639248251"
Jul 26 17:49:09.411587 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:09.411551    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af7dc810-9c37-4155-ab59-7c4196ffb50b-metrics-certs\") pod \"network-metrics-daemon-7mtnm\" (UID: \"af7dc810-9c37-4155-ab59-7c4196ffb50b\") " pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:09.411762 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:09.411622    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32cbd054-d7a7-4c00-be2e-012e16598a49-original-pull-secret\") pod \"global-pull-secret-syncer-96jft\" (UID: \"32cbd054-d7a7-4c00-be2e-012e16598a49\") " pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:09.411842 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:09.411770    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 17:49:09.411842 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:09.411828    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32cbd054-d7a7-4c00-be2e-012e16598a49-original-pull-secret podName:32cbd054-d7a7-4c00-be2e-012e16598a49 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:13.411810857 +0000 UTC m=+10.064001715 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/32cbd054-d7a7-4c00-be2e-012e16598a49-original-pull-secret") pod "global-pull-secret-syncer-96jft" (UID: "32cbd054-d7a7-4c00-be2e-012e16598a49") : object "kube-system"/"original-pull-secret" not registered
Jul 26 17:49:09.412252 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:09.412229    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:49:09.412349 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:09.412287    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af7dc810-9c37-4155-ab59-7c4196ffb50b-metrics-certs podName:af7dc810-9c37-4155-ab59-7c4196ffb50b nodeName:}" failed. No retries permitted until 2026-07-26 17:49:13.412271235 +0000 UTC m=+10.064462084 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af7dc810-9c37-4155-ab59-7c4196ffb50b-metrics-certs") pod "network-metrics-daemon-7mtnm" (UID: "af7dc810-9c37-4155-ab59-7c4196ffb50b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:49:09.512731 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:09.512296    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jntqg\" (UniqueName: \"kubernetes.io/projected/76a782a1-4344-459a-8f5c-3e84c018f0f7-kube-api-access-jntqg\") pod \"network-check-target-h8sjb\" (UID: \"76a782a1-4344-459a-8f5c-3e84c018f0f7\") " pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:09.512731 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:09.512452    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 17:49:09.512731 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:09.512475    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 17:49:09.512731 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:09.512486    2570 projected.go:196] Error preparing data for projected volume kube-api-access-jntqg for pod openshift-network-diagnostics/network-check-target-h8sjb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:49:09.512731 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:09.512536    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76a782a1-4344-459a-8f5c-3e84c018f0f7-kube-api-access-jntqg podName:76a782a1-4344-459a-8f5c-3e84c018f0f7 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:13.51252417 +0000 UTC m=+10.164715020 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jntqg" (UniqueName: "kubernetes.io/projected/76a782a1-4344-459a-8f5c-3e84c018f0f7-kube-api-access-jntqg") pod "network-check-target-h8sjb" (UID: "76a782a1-4344-459a-8f5c-3e84c018f0f7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:49:09.835379 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:09.834861    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:09.835379 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:09.835011    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtnm" podUID="af7dc810-9c37-4155-ab59-7c4196ffb50b"
Jul 26 17:49:09.835379 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:09.835124    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:09.835379 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:09.835233    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96jft" podUID="32cbd054-d7a7-4c00-be2e-012e16598a49"
Jul 26 17:49:09.835379 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:09.835280    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:09.835379 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:09.835342    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h8sjb" podUID="76a782a1-4344-459a-8f5c-3e84c018f0f7"
Jul 26 17:49:11.832181 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:11.832145    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:11.832181 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:11.832185    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:11.832674 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:11.832297    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtnm" podUID="af7dc810-9c37-4155-ab59-7c4196ffb50b"
Jul 26 17:49:11.832674 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:11.832664    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96jft" podUID="32cbd054-d7a7-4c00-be2e-012e16598a49"
Jul 26 17:49:11.832800 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:11.832716    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:11.832800 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:11.832776    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h8sjb" podUID="76a782a1-4344-459a-8f5c-3e84c018f0f7"
Jul 26 17:49:13.440141 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:13.440091    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af7dc810-9c37-4155-ab59-7c4196ffb50b-metrics-certs\") pod \"network-metrics-daemon-7mtnm\" (UID: \"af7dc810-9c37-4155-ab59-7c4196ffb50b\") " pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:13.440624 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:13.440176    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32cbd054-d7a7-4c00-be2e-012e16598a49-original-pull-secret\") pod \"global-pull-secret-syncer-96jft\" (UID: \"32cbd054-d7a7-4c00-be2e-012e16598a49\") " pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:13.440624 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:13.440304    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 17:49:13.440624 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:13.440363    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32cbd054-d7a7-4c00-be2e-012e16598a49-original-pull-secret podName:32cbd054-d7a7-4c00-be2e-012e16598a49 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:21.440345749 +0000 UTC m=+18.092536588 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/32cbd054-d7a7-4c00-be2e-012e16598a49-original-pull-secret") pod "global-pull-secret-syncer-96jft" (UID: "32cbd054-d7a7-4c00-be2e-012e16598a49") : object "kube-system"/"original-pull-secret" not registered
Jul 26 17:49:13.440781 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:13.440762    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:49:13.440838 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:13.440822    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af7dc810-9c37-4155-ab59-7c4196ffb50b-metrics-certs podName:af7dc810-9c37-4155-ab59-7c4196ffb50b nodeName:}" failed. No retries permitted until 2026-07-26 17:49:21.440805372 +0000 UTC m=+18.092996208 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af7dc810-9c37-4155-ab59-7c4196ffb50b-metrics-certs") pod "network-metrics-daemon-7mtnm" (UID: "af7dc810-9c37-4155-ab59-7c4196ffb50b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:49:13.541450 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:13.541404    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jntqg\" (UniqueName: \"kubernetes.io/projected/76a782a1-4344-459a-8f5c-3e84c018f0f7-kube-api-access-jntqg\") pod \"network-check-target-h8sjb\" (UID: \"76a782a1-4344-459a-8f5c-3e84c018f0f7\") " pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:13.541612 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:13.541591    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 17:49:13.541672 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:13.541618    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 17:49:13.541672 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:13.541628    2570 projected.go:196] Error preparing data for projected volume kube-api-access-jntqg for pod openshift-network-diagnostics/network-check-target-h8sjb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:49:13.541759 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:13.541685    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76a782a1-4344-459a-8f5c-3e84c018f0f7-kube-api-access-jntqg podName:76a782a1-4344-459a-8f5c-3e84c018f0f7 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:21.541666947 +0000 UTC m=+18.193857802 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jntqg" (UniqueName: "kubernetes.io/projected/76a782a1-4344-459a-8f5c-3e84c018f0f7-kube-api-access-jntqg") pod "network-check-target-h8sjb" (UID: "76a782a1-4344-459a-8f5c-3e84c018f0f7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:49:13.833457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:13.833426    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:13.833653 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:13.833545    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtnm" podUID="af7dc810-9c37-4155-ab59-7c4196ffb50b"
Jul 26 17:49:13.833954 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:13.833925    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:13.834224 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:13.834198    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96jft" podUID="32cbd054-d7a7-4c00-be2e-012e16598a49"
Jul 26 17:49:13.834477 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:13.834458    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:13.834674 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:13.834653    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h8sjb" podUID="76a782a1-4344-459a-8f5c-3e84c018f0f7"
Jul 26 17:49:15.832660 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:15.832618    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:15.833056 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:15.832751    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h8sjb" podUID="76a782a1-4344-459a-8f5c-3e84c018f0f7"
Jul 26 17:49:15.833056 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:15.832893    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:15.833056 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:15.832986    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtnm" podUID="af7dc810-9c37-4155-ab59-7c4196ffb50b"
Jul 26 17:49:15.833225 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:15.833074    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:15.833225 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:15.833158    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96jft" podUID="32cbd054-d7a7-4c00-be2e-012e16598a49"
Jul 26 17:49:17.832100 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:17.831990    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:17.832100 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:17.831995    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:17.832604 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:17.832136    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:17.832604 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:17.832147    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96jft" podUID="32cbd054-d7a7-4c00-be2e-012e16598a49"
Jul 26 17:49:17.832604 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:17.832235    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h8sjb" podUID="76a782a1-4344-459a-8f5c-3e84c018f0f7"
Jul 26 17:49:17.832604 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:17.832324    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtnm" podUID="af7dc810-9c37-4155-ab59-7c4196ffb50b"
Jul 26 17:49:19.832498 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:19.832425    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:19.833006 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:19.832426    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:19.833006 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:19.832544    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:19.833006 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:19.832549    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h8sjb" podUID="76a782a1-4344-459a-8f5c-3e84c018f0f7"
Jul 26 17:49:19.833006 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:19.832625    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96jft" podUID="32cbd054-d7a7-4c00-be2e-012e16598a49"
Jul 26 17:49:19.833006 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:19.832720    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtnm" podUID="af7dc810-9c37-4155-ab59-7c4196ffb50b"
Jul 26 17:49:21.502757 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:21.502716    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af7dc810-9c37-4155-ab59-7c4196ffb50b-metrics-certs\") pod \"network-metrics-daemon-7mtnm\" (UID: \"af7dc810-9c37-4155-ab59-7c4196ffb50b\") " pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:21.503239 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:21.502897    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:49:21.503239 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:21.502901    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32cbd054-d7a7-4c00-be2e-012e16598a49-original-pull-secret\") pod \"global-pull-secret-syncer-96jft\" (UID: \"32cbd054-d7a7-4c00-be2e-012e16598a49\") " pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:21.503239 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:21.502968    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af7dc810-9c37-4155-ab59-7c4196ffb50b-metrics-certs podName:af7dc810-9c37-4155-ab59-7c4196ffb50b nodeName:}" failed. No retries permitted until 2026-07-26 17:49:37.502946041 +0000 UTC m=+34.155136892 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af7dc810-9c37-4155-ab59-7c4196ffb50b-metrics-certs") pod "network-metrics-daemon-7mtnm" (UID: "af7dc810-9c37-4155-ab59-7c4196ffb50b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:49:21.503239 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:21.503005    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 17:49:21.503239 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:21.503062    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32cbd054-d7a7-4c00-be2e-012e16598a49-original-pull-secret podName:32cbd054-d7a7-4c00-be2e-012e16598a49 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:37.50304565 +0000 UTC m=+34.155236486 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/32cbd054-d7a7-4c00-be2e-012e16598a49-original-pull-secret") pod "global-pull-secret-syncer-96jft" (UID: "32cbd054-d7a7-4c00-be2e-012e16598a49") : object "kube-system"/"original-pull-secret" not registered
Jul 26 17:49:21.604341 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:21.604297    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jntqg\" (UniqueName: \"kubernetes.io/projected/76a782a1-4344-459a-8f5c-3e84c018f0f7-kube-api-access-jntqg\") pod \"network-check-target-h8sjb\" (UID: \"76a782a1-4344-459a-8f5c-3e84c018f0f7\") " pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:21.604505 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:21.604462    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 17:49:21.604505 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:21.604492    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 17:49:21.604588 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:21.604507    2570 projected.go:196] Error preparing data for projected volume kube-api-access-jntqg for pod openshift-network-diagnostics/network-check-target-h8sjb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:49:21.604588 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:21.604571    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76a782a1-4344-459a-8f5c-3e84c018f0f7-kube-api-access-jntqg podName:76a782a1-4344-459a-8f5c-3e84c018f0f7 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:37.604551496 +0000 UTC m=+34.256742350 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jntqg" (UniqueName: "kubernetes.io/projected/76a782a1-4344-459a-8f5c-3e84c018f0f7-kube-api-access-jntqg") pod "network-check-target-h8sjb" (UID: "76a782a1-4344-459a-8f5c-3e84c018f0f7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:49:21.832185 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:21.832151    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:21.832185 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:21.832190    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:21.832403 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:21.832151    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:21.832403 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:21.832267    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h8sjb" podUID="76a782a1-4344-459a-8f5c-3e84c018f0f7"
Jul 26 17:49:21.832403 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:21.832370    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96jft" podUID="32cbd054-d7a7-4c00-be2e-012e16598a49"
Jul 26 17:49:21.832608 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:21.832480    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtnm" podUID="af7dc810-9c37-4155-ab59-7c4196ffb50b"
Jul 26 17:49:23.832592 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:23.832560    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:23.832592 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:23.832599    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:23.833132 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:23.832675    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtnm" podUID="af7dc810-9c37-4155-ab59-7c4196ffb50b"
Jul 26 17:49:23.833132 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:23.832713    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96jft" podUID="32cbd054-d7a7-4c00-be2e-012e16598a49"
Jul 26 17:49:23.833132 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:23.832747    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:23.833132 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:23.832809    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h8sjb" podUID="76a782a1-4344-459a-8f5c-3e84c018f0f7"
Jul 26 17:49:25.833098 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:25.832821    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:25.833848 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:25.832820    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:25.833848 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:25.832820    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:25.833848 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:25.833493    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h8sjb" podUID="76a782a1-4344-459a-8f5c-3e84c018f0f7"
Jul 26 17:49:25.833848 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:25.833652    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96jft" podUID="32cbd054-d7a7-4c00-be2e-012e16598a49"
Jul 26 17:49:25.833848 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:25.833801    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtnm" podUID="af7dc810-9c37-4155-ab59-7c4196ffb50b"
Jul 26 17:49:26.005650 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:26.005563    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c" event={"ID":"b2473540-1734-4f1a-ab9a-719694dd5c96","Type":"ContainerStarted","Data":"2d624f3699b29827059aa360a37983b4bb4b67533feab2ca8ec25d02c96ba0bf"}
Jul 26 17:49:26.006854 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:26.006832    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-khbbn" event={"ID":"3514ccab-ab9c-46b1-bf1a-21cac46c7586","Type":"ContainerStarted","Data":"8e9f830e147d52a868b0737280b4bb48af3f0da559cc32460430fc23135af962"}
Jul 26 17:49:26.008275 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:26.008253    2570 generic.go:356] "Generic (PLEG): container finished" podID="a337e0eb-4d13-4df5-bfe7-186de89d4dcc" containerID="03b8767c4b8849dbda64209f754d0a05978d48cd3ee133aaf006535529f25be1" exitCode=0
Jul 26 17:49:26.008361 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:26.008330    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4zzb" event={"ID":"a337e0eb-4d13-4df5-bfe7-186de89d4dcc","Type":"ContainerDied","Data":"03b8767c4b8849dbda64209f754d0a05978d48cd3ee133aaf006535529f25be1"}
Jul 26 17:49:26.009727 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:26.009701    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-974ql" event={"ID":"1fcdab4c-350c-44c3-9bc0-a9995d33c539","Type":"ContainerStarted","Data":"4ecebc415ed05ec245a96c22aefe4225bd8d49367fff054ba277bb5a63e54ee9"}
Jul 26 17:49:26.011206 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:26.011189    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ccd9g" event={"ID":"048c9bc9-3ff2-4667-8392-103e4e235f0a","Type":"ContainerStarted","Data":"d908b57c9e1c9130886e3c2525c483e71014096da40b337ad3d6f45c8f48f4e0"}
Jul 26 17:49:26.012503 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:26.012483    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ddfsj" event={"ID":"0557c247-c431-4eef-9de0-0b272b9e5250","Type":"ContainerStarted","Data":"e60f316c7ace14b5d66cbe7d8bf3b82c568e227ca179a5b35fe9ca0905f20897"}
Jul 26 17:49:26.015693 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:26.015674    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5zh25" event={"ID":"3f795165-d9b9-4634-a24e-d10cee628e00","Type":"ContainerStarted","Data":"c98e115d827ef0c007edee90fc28768076507e5c8d037a6d7707ea4739929786"}
Jul 26 17:49:26.015778 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:26.015731    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5zh25" event={"ID":"3f795165-d9b9-4634-a24e-d10cee628e00","Type":"ContainerStarted","Data":"8e2b5f02fae4d4644d7f78700d8485a8371ea65385f999561384fd3aa9edf0a0"}
Jul 26 17:49:26.015827 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:26.015778    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5zh25" event={"ID":"3f795165-d9b9-4634-a24e-d10cee628e00","Type":"ContainerStarted","Data":"ab2b2f0094c5ad144e14448729d2ae33a28e525d1186ad5099613b05f4ff7978"}
Jul 26 17:49:26.015827 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:26.015816    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5zh25" event={"ID":"3f795165-d9b9-4634-a24e-d10cee628e00","Type":"ContainerStarted","Data":"baf0959f542f00983a5c6002d788c893f0ef2167098d091f7a04f678a0c7f91d"}
Jul 26 17:49:26.015916 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:26.015854    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5zh25" event={"ID":"3f795165-d9b9-4634-a24e-d10cee628e00","Type":"ContainerStarted","Data":"de1d798b1fe23f075b1c55cf5aaf57cb3c962b82e62da4aa79e2a9ba7db55bbe"}
Jul 26 17:49:26.015968 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:26.015903    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5zh25" event={"ID":"3f795165-d9b9-4634-a24e-d10cee628e00","Type":"ContainerStarted","Data":"cc8a6d6eaf37cdbca1fd1b622aedbd82a709a27b1d41528093c97520a3a2072b"}
Jul 26 17:49:26.016950 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:26.016932    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tslqc" event={"ID":"63076f7b-11ca-42d6-8d7b-2e5a96035317","Type":"ContainerStarted","Data":"ef1221c1ea84715da7e65e0e68b909d3c8786997c1c71981e183f9c9805c6d44"}
Jul 26 17:49:26.021501 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:26.021466    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-khbbn" podStartSLOduration=11.614657703 podStartE2EDuration="22.021455328s" totalImagesPullingTime="10.406797625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:49:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:26.021237173 +0000 UTC m=+22.673428031" watchObservedRunningTime="2026-07-26 17:49:26.021455328 +0000 UTC m=+22.673646186"
Jul 26 17:49:26.055313 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:26.055262    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-974ql" podStartSLOduration=3.804321889 podStartE2EDuration="22.055245006s" totalImagesPullingTime="18.250923117s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:49:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:26.055211092 +0000 UTC m=+22.707401965" watchObservedRunningTime="2026-07-26 17:49:26.055245006 +0000 UTC m=+22.707435879"
Jul 26 17:49:26.069467 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:26.069428    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ddfsj" podStartSLOduration=3.814180027 podStartE2EDuration="22.069410027s" totalImagesPullingTime="18.25523s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:49:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:26.068862691 +0000 UTC m=+22.721053549" watchObservedRunningTime="2026-07-26 17:49:26.069410027 +0000 UTC m=+22.721600886"
Jul 26 17:49:26.086983 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:26.086941    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tslqc" podStartSLOduration=3.7834274629999998 podStartE2EDuration="22.086928641s" totalImagesPullingTime="18.303501178s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:49:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:26.086513846 +0000 UTC m=+22.738704702" watchObservedRunningTime="2026-07-26 17:49:26.086928641 +0000 UTC m=+22.739119498"
Jul 26 17:49:26.106185 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:26.106141    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ccd9g" podStartSLOduration=3.851434476 podStartE2EDuration="22.106103009s" totalImagesPullingTime="18.254668533s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:49:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:26.105559929 +0000 UTC m=+22.757750786" watchObservedRunningTime="2026-07-26 17:49:26.106103009 +0000 UTC m=+22.758293869"
Jul 26 17:49:26.485945 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:26.485912    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-khbbn"
Jul 26 17:49:26.877363 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:26.877340    2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 26 17:49:27.021353 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:27.021319    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8sbv6" event={"ID":"fcc71092-cd57-404f-aa06-8a3d8486eb33","Type":"ContainerStarted","Data":"c704599a7e13fd96600603fd09e2946e93e066db83a539afa1827f778df49fca"}
Jul 26 17:49:27.023512 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:27.023453    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c" event={"ID":"b2473540-1734-4f1a-ab9a-719694dd5c96","Type":"ContainerStarted","Data":"d39e1d5a90563e05433d2794aa97dcd867fc50af6ac3cdb7a81d1439236e82e3"}
Jul 26 17:49:27.038294 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:27.038242    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8sbv6" podStartSLOduration=4.783844279 podStartE2EDuration="23.038220926s" totalImagesPullingTime="18.254376647s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:49:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:27.037696758 +0000 UTC m=+23.689887616" watchObservedRunningTime="2026-07-26 17:49:27.038220926 +0000 UTC m=+23.690411785"
Jul 26 17:49:27.827908 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:27.827792    2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-26T17:49:26.877357078Z","UUID":"ed339245-e23b-4134-abea-50f11968ed26","Handler":null,"Name":"","Endpoint":""}
Jul 26 17:49:27.829664 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:27.829641    2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 26 17:49:27.829789 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:27.829686    2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 26 17:49:27.832294 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:27.832274    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:27.832447 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:27.832388    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96jft" podUID="32cbd054-d7a7-4c00-be2e-012e16598a49"
Jul 26 17:49:27.832616 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:27.832522    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:27.832684 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:27.832626    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h8sjb" podUID="76a782a1-4344-459a-8f5c-3e84c018f0f7"
Jul 26 17:49:27.832764 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:27.832744    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:27.832841 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:27.832823    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtnm" podUID="af7dc810-9c37-4155-ab59-7c4196ffb50b"
Jul 26 17:49:28.967597 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:28.967377    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-khbbn"
Jul 26 17:49:28.968061 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:28.968042    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-khbbn"
Jul 26 17:49:29.030978 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:29.030943    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5zh25" event={"ID":"3f795165-d9b9-4634-a24e-d10cee628e00","Type":"ContainerStarted","Data":"f351ff3f8ac019c9dc2851f35fbe657ded0048d396414b0db44f6c9b664272fe"}
Jul 26 17:49:29.033257 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:29.033232    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c" event={"ID":"b2473540-1734-4f1a-ab9a-719694dd5c96","Type":"ContainerStarted","Data":"d2a688b83ac92279041cad6479b5566d8b3258bcdc669bc8cdf0fda4a2afc516"}
Jul 26 17:49:29.033885 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:29.033861    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-khbbn"
Jul 26 17:49:29.049787 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:29.049724    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbf4c" podStartSLOduration=4.04307288 podStartE2EDuration="25.049709659s" totalImagesPullingTime="21.006636779s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:49:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:29.049359182 +0000 UTC m=+25.701550041" watchObservedRunningTime="2026-07-26 17:49:29.049709659 +0000 UTC m=+25.701900517"
Jul 26 17:49:29.832495 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:29.832463    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:29.832495 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:29.832494    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:29.832738 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:29.832463    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:29.832738 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:29.832609    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96jft" podUID="32cbd054-d7a7-4c00-be2e-012e16598a49"
Jul 26 17:49:29.832738 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:29.832678    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtnm" podUID="af7dc810-9c37-4155-ab59-7c4196ffb50b"
Jul 26 17:49:29.832900 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:29.832746    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h8sjb" podUID="76a782a1-4344-459a-8f5c-3e84c018f0f7"
Jul 26 17:49:31.040013 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:31.039812    2570 generic.go:356] "Generic (PLEG): container finished" podID="a337e0eb-4d13-4df5-bfe7-186de89d4dcc" containerID="d06eb91d21eb4be6ea3b495124272418eadff3f1676123f11023ed6a3fb75990" exitCode=0
Jul 26 17:49:31.040891 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:31.039894    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4zzb" event={"ID":"a337e0eb-4d13-4df5-bfe7-186de89d4dcc","Type":"ContainerDied","Data":"d06eb91d21eb4be6ea3b495124272418eadff3f1676123f11023ed6a3fb75990"}
Jul 26 17:49:31.043783 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:31.043759    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5zh25" event={"ID":"3f795165-d9b9-4634-a24e-d10cee628e00","Type":"ContainerStarted","Data":"a1ef64592dd4e168e3b2dcf1159a12e69191bbea053025b3d6001154eded640d"}
Jul 26 17:49:31.085767 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:31.085719    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5zh25" podStartSLOduration=8.764941179000001 podStartE2EDuration="27.085706073s" totalImagesPullingTime="18.320764894s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:49:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:31.085301129 +0000 UTC m=+27.737491986" watchObservedRunningTime="2026-07-26 17:49:31.085706073 +0000 UTC m=+27.737896930"
Jul 26 17:49:31.831980 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:31.831949    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:31.832212 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:31.832001    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:31.832212 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:31.831949    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:31.832212 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:31.832083    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtnm" podUID="af7dc810-9c37-4155-ab59-7c4196ffb50b"
Jul 26 17:49:31.832212 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:31.832177    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96jft" podUID="32cbd054-d7a7-4c00-be2e-012e16598a49"
Jul 26 17:49:31.832370 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:31.832255    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h8sjb" podUID="76a782a1-4344-459a-8f5c-3e84c018f0f7"
Jul 26 17:49:32.046022 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:32.045996    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:32.046468 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:32.046079    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:32.046468 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:32.046141    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:32.061598 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:32.061571    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:32.061750 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:32.061674    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:49:32.523312 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:32.523281    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-96jft"]
Jul 26 17:49:32.523491 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:32.523458    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:32.523598 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:32.523572    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96jft" podUID="32cbd054-d7a7-4c00-be2e-012e16598a49"
Jul 26 17:49:32.526598 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:32.526570    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7mtnm"]
Jul 26 17:49:32.526738 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:32.526695    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:32.526827 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:32.526808    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtnm" podUID="af7dc810-9c37-4155-ab59-7c4196ffb50b"
Jul 26 17:49:32.527189 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:32.527155    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-h8sjb"]
Jul 26 17:49:32.527302 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:32.527254    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:32.527362 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:32.527337    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h8sjb" podUID="76a782a1-4344-459a-8f5c-3e84c018f0f7"
Jul 26 17:49:33.049854 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:33.049819    2570 generic.go:356] "Generic (PLEG): container finished" podID="a337e0eb-4d13-4df5-bfe7-186de89d4dcc" containerID="5535d4cad91a3a27427b9cbd6db72243fe2f051fb5f916d0af75497e60477928" exitCode=0
Jul 26 17:49:33.050313 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:33.049907    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4zzb" event={"ID":"a337e0eb-4d13-4df5-bfe7-186de89d4dcc","Type":"ContainerDied","Data":"5535d4cad91a3a27427b9cbd6db72243fe2f051fb5f916d0af75497e60477928"}
Jul 26 17:49:33.833490 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:33.833457    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:33.833677 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:33.833571    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtnm" podUID="af7dc810-9c37-4155-ab59-7c4196ffb50b"
Jul 26 17:49:33.833677 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:33.833631    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:33.833789 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:33.833720    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h8sjb" podUID="76a782a1-4344-459a-8f5c-3e84c018f0f7"
Jul 26 17:49:34.832499 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:34.832459    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:34.832860 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:34.832570    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96jft" podUID="32cbd054-d7a7-4c00-be2e-012e16598a49"
Jul 26 17:49:35.056753 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:35.056578    2570 generic.go:356] "Generic (PLEG): container finished" podID="a337e0eb-4d13-4df5-bfe7-186de89d4dcc" containerID="06e624132e2a24dce27074f66a20bdb8cf8969b0228b7acacb1e933285a31ecd" exitCode=0
Jul 26 17:49:35.056753 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:35.056656    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4zzb" event={"ID":"a337e0eb-4d13-4df5-bfe7-186de89d4dcc","Type":"ContainerDied","Data":"06e624132e2a24dce27074f66a20bdb8cf8969b0228b7acacb1e933285a31ecd"}
Jul 26 17:49:35.832647 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:35.832616    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:35.833089 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:35.832617    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:35.833089 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:35.832760    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtnm" podUID="af7dc810-9c37-4155-ab59-7c4196ffb50b"
Jul 26 17:49:35.833089 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:35.832820    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h8sjb" podUID="76a782a1-4344-459a-8f5c-3e84c018f0f7"
Jul 26 17:49:36.832895 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:36.832858    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:36.833316 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:36.832993    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96jft" podUID="32cbd054-d7a7-4c00-be2e-012e16598a49"
Jul 26 17:49:37.520990 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:37.520948    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af7dc810-9c37-4155-ab59-7c4196ffb50b-metrics-certs\") pod \"network-metrics-daemon-7mtnm\" (UID: \"af7dc810-9c37-4155-ab59-7c4196ffb50b\") " pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:37.521190 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:37.521024    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32cbd054-d7a7-4c00-be2e-012e16598a49-original-pull-secret\") pod \"global-pull-secret-syncer-96jft\" (UID: \"32cbd054-d7a7-4c00-be2e-012e16598a49\") " pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:37.521190 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:37.521108    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:49:37.521190 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:37.521162    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 17:49:37.521312 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:37.521198    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af7dc810-9c37-4155-ab59-7c4196ffb50b-metrics-certs podName:af7dc810-9c37-4155-ab59-7c4196ffb50b nodeName:}" failed. No retries permitted until 2026-07-26 17:50:09.5211818 +0000 UTC m=+66.173372636 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af7dc810-9c37-4155-ab59-7c4196ffb50b-metrics-certs") pod "network-metrics-daemon-7mtnm" (UID: "af7dc810-9c37-4155-ab59-7c4196ffb50b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:49:37.521312 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:37.521213    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32cbd054-d7a7-4c00-be2e-012e16598a49-original-pull-secret podName:32cbd054-d7a7-4c00-be2e-012e16598a49 nodeName:}" failed. No retries permitted until 2026-07-26 17:50:09.521206584 +0000 UTC m=+66.173397420 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/32cbd054-d7a7-4c00-be2e-012e16598a49-original-pull-secret") pod "global-pull-secret-syncer-96jft" (UID: "32cbd054-d7a7-4c00-be2e-012e16598a49") : object "kube-system"/"original-pull-secret" not registered
Jul 26 17:49:37.622384 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:37.622279    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jntqg\" (UniqueName: \"kubernetes.io/projected/76a782a1-4344-459a-8f5c-3e84c018f0f7-kube-api-access-jntqg\") pod \"network-check-target-h8sjb\" (UID: \"76a782a1-4344-459a-8f5c-3e84c018f0f7\") " pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:37.622578 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:37.622500    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 17:49:37.622578 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:37.622531    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 17:49:37.622578 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:37.622545    2570 projected.go:196] Error preparing data for projected volume kube-api-access-jntqg for pod openshift-network-diagnostics/network-check-target-h8sjb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:49:37.622751 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:37.622602    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76a782a1-4344-459a-8f5c-3e84c018f0f7-kube-api-access-jntqg podName:76a782a1-4344-459a-8f5c-3e84c018f0f7 nodeName:}" failed. No retries permitted until 2026-07-26 17:50:09.622584528 +0000 UTC m=+66.274775385 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jntqg" (UniqueName: "kubernetes.io/projected/76a782a1-4344-459a-8f5c-3e84c018f0f7-kube-api-access-jntqg") pod "network-check-target-h8sjb" (UID: "76a782a1-4344-459a-8f5c-3e84c018f0f7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:49:37.832515 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:37.832430    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:37.832515 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:37.832482    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:37.832729 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:37.832582    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mtnm" podUID="af7dc810-9c37-4155-ab59-7c4196ffb50b"
Jul 26 17:49:37.832729 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:37.832709    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h8sjb" podUID="76a782a1-4344-459a-8f5c-3e84c018f0f7"
Jul 26 17:49:38.831996 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:38.831963    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:38.832569 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:38.832091    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96jft" podUID="32cbd054-d7a7-4c00-be2e-012e16598a49"
Jul 26 17:49:39.163983 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.163961    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-45.ec2.internal" event="NodeReady"
Jul 26 17:49:39.164132 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.164077    2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 26 17:49:39.197776 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.197744    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-f9b95bd64-5jvnp"]
Jul 26 17:49:39.202640 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.202606    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hsdvj"]
Jul 26 17:49:39.202771 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.202742    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.205608 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.205585    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2hr9p\""
Jul 26 17:49:39.205847 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.205832    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 26 17:49:39.206154 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.206130    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qdq4z"]
Jul 26 17:49:39.206239 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.206216    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hsdvj"
Jul 26 17:49:39.206600 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.206575    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 26 17:49:39.208187 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.208142    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 26 17:49:39.208934 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.208917    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-n8dcr\""
Jul 26 17:49:39.209580 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.209562    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-q8vbm"]
Jul 26 17:49:39.209713 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.209694    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qdq4z"
Jul 26 17:49:39.212085 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.212066    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 26 17:49:39.212417 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.212385    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wkmvr\""
Jul 26 17:49:39.212524 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.212428    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 26 17:49:39.212524 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.212466    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 26 17:49:39.212524 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.212466    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 26 17:49:39.212715 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.212692    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-577j2"]
Jul 26 17:49:39.212822 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.212802    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm"
Jul 26 17:49:39.212822 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.212815    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 26 17:49:39.213026 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.212904    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 26 17:49:39.215784 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.215766    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc"]
Jul 26 17:49:39.215921 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.215901    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-577j2"
Jul 26 17:49:39.216577 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.216558    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-b6ft8\""
Jul 26 17:49:39.216787 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.216773    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 26 17:49:39.216842 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.216801    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 26 17:49:39.216998 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.216986    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 26 17:49:39.217038 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.217025    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 26 17:49:39.219989 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.219966    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc"
Jul 26 17:49:39.226278 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.222843    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-bqgpr\""
Jul 26 17:49:39.226278 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.223451    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 26 17:49:39.226278 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.224482    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 26 17:49:39.227225 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.227202    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 26 17:49:39.228078 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.227708    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 26 17:49:39.231793 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.231771    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 26 17:49:39.231874 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.231774    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f9b95bd64-5jvnp"]
Jul 26 17:49:39.231933 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.231901    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hsdvj"]
Jul 26 17:49:39.232327 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.232304    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 26 17:49:39.232426 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.232384    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-577j2"]
Jul 26 17:49:39.232700 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.232680    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-28xsb\""
Jul 26 17:49:39.232786 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.232707    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 26 17:49:39.233012 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.232993    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-z2knl"]
Jul 26 17:49:39.233126 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.233099    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 26 17:49:39.233191 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.233179    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 26 17:49:39.236307 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.236291    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc"]
Jul 26 17:49:39.236424 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.236316    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qdq4z"]
Jul 26 17:49:39.236424 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.236333    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-q8vbm"]
Jul 26 17:49:39.236424 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.236348    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-c9hsm"]
Jul 26 17:49:39.236675 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.236438    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-z2knl"
Jul 26 17:49:39.238955 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.238929    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 26 17:49:39.239057 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.238934    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 26 17:49:39.239407 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.239386    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-r6mlm\""
Jul 26 17:49:39.239714 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.239690    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-26rxm"]
Jul 26 17:49:39.240692 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.239827    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c9hsm"
Jul 26 17:49:39.242617 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.242595    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 26 17:49:39.242907 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.242819    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 26 17:49:39.243407 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.243019    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 26 17:49:39.243407 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.243208    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-btq2h\""
Jul 26 17:49:39.244582 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.244211    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-c9hsm"]
Jul 26 17:49:39.244582 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.244246    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-z2knl"]
Jul 26 17:49:39.244582 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.244341    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-26rxm"
Jul 26 17:49:39.244582 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.244432    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-26rxm"]
Jul 26 17:49:39.246535 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.246470    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 26 17:49:39.246535 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.246478    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 26 17:49:39.246709 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.246560    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jb7dx\""
Jul 26 17:49:39.334155 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334125    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5szt\" (UniqueName: \"kubernetes.io/projected/5b18e275-81da-4553-921e-b95957e34eb6-kube-api-access-h5szt\") pod \"network-check-source-7d88cb55d8-z2knl\" (UID: \"5b18e275-81da-4553-921e-b95957e34eb6\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-z2knl"
Jul 26 17:49:39.334259 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334161    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f532906-dddd-459d-b8da-3074eab679b1-installation-pull-secrets\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.334259 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334198    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/581ed0b5-91f3-4930-adfe-379ab913be79-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-qdq4z\" (UID: \"581ed0b5-91f3-4930-adfe-379ab913be79\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qdq4z"
Jul 26 17:49:39.334259 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334235    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ms2v\" (UniqueName: \"kubernetes.io/projected/40cb8009-e6c4-4fc2-a574-c4ecca296db4-kube-api-access-7ms2v\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc\" (UID: \"40cb8009-e6c4-4fc2-a574-c4ecca296db4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc"
Jul 26 17:49:39.334259 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334253    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwr9f\" (UniqueName: \"kubernetes.io/projected/e6cff45e-fc7b-46bd-a628-a5e5b767269a-kube-api-access-wwr9f\") pod \"service-ca-operator-5d6c75f84f-577j2\" (UID: \"e6cff45e-fc7b-46bd-a628-a5e5b767269a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-577j2"
Jul 26 17:49:39.334381 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334271    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f532906-dddd-459d-b8da-3074eab679b1-image-registry-private-configuration\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.334381 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334309    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c07c88c4-6137-4a7e-b0ea-4e266eb50761-tmp-dir\") pod \"dns-default-26rxm\" (UID: \"c07c88c4-6137-4a7e-b0ea-4e266eb50761\") " pod="openshift-dns/dns-default-26rxm"
Jul 26 17:49:39.334381 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334364    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lckds\" (UniqueName: \"kubernetes.io/projected/801b12f8-cc05-4646-99a0-304297466b26-kube-api-access-lckds\") pod \"console-operator-584dcb6b54-q8vbm\" (UID: \"801b12f8-cc05-4646-99a0-304297466b26\") " pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm"
Jul 26 17:49:39.334469 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334384    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/40cb8009-e6c4-4fc2-a574-c4ecca296db4-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc\" (UID: \"40cb8009-e6c4-4fc2-a574-c4ecca296db4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc"
Jul 26 17:49:39.334469 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334400    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/801b12f8-cc05-4646-99a0-304297466b26-serving-cert\") pod \"console-operator-584dcb6b54-q8vbm\" (UID: \"801b12f8-cc05-4646-99a0-304297466b26\") " pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm"
Jul 26 17:49:39.334469 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334434    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-registry-tls\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.334469 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334461    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f532906-dddd-459d-b8da-3074eab679b1-ca-trust-extracted\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.334583 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334477    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3321b850-63be-4345-ac65-afaee093369a-cert\") pod \"ingress-canary-c9hsm\" (UID: \"3321b850-63be-4345-ac65-afaee093369a\") " pod="openshift-ingress-canary/ingress-canary-c9hsm"
Jul 26 17:49:39.334583 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334492    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-475zf\" (UniqueName: \"kubernetes.io/projected/c07c88c4-6137-4a7e-b0ea-4e266eb50761-kube-api-access-475zf\") pod \"dns-default-26rxm\" (UID: \"c07c88c4-6137-4a7e-b0ea-4e266eb50761\") " pod="openshift-dns/dns-default-26rxm"
Jul 26 17:49:39.334583 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334519    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v89p\" (UniqueName: \"kubernetes.io/projected/51d5a4bf-5978-4bb0-81e8-3004f8519805-kube-api-access-6v89p\") pod \"volume-data-source-validator-549c6d54cf-hsdvj\" (UID: \"51d5a4bf-5978-4bb0-81e8-3004f8519805\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hsdvj"
Jul 26 17:49:39.334583 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334537    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/40cb8009-e6c4-4fc2-a574-c4ecca296db4-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc\" (UID: \"40cb8009-e6c4-4fc2-a574-c4ecca296db4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc"
Jul 26 17:49:39.334583 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334552    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c07c88c4-6137-4a7e-b0ea-4e266eb50761-metrics-tls\") pod \"dns-default-26rxm\" (UID: \"c07c88c4-6137-4a7e-b0ea-4e266eb50761\") " pod="openshift-dns/dns-default-26rxm"
Jul 26 17:49:39.334583 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334567    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-bound-sa-token\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.334739 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334585    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f532906-dddd-459d-b8da-3074eab679b1-registry-certificates\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.334739 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334619    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e6cff45e-fc7b-46bd-a628-a5e5b767269a-serving-cert\") pod \"service-ca-operator-5d6c75f84f-577j2\" (UID: \"e6cff45e-fc7b-46bd-a628-a5e5b767269a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-577j2"
Jul 26 17:49:39.334739 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334646    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82q2n\" (UniqueName: \"kubernetes.io/projected/3321b850-63be-4345-ac65-afaee093369a-kube-api-access-82q2n\") pod \"ingress-canary-c9hsm\" (UID: \"3321b850-63be-4345-ac65-afaee093369a\") " pod="openshift-ingress-canary/ingress-canary-c9hsm"
Jul 26 17:49:39.334739 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334672    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e6cff45e-fc7b-46bd-a628-a5e5b767269a-config\") pod \"service-ca-operator-5d6c75f84f-577j2\" (UID: \"e6cff45e-fc7b-46bd-a628-a5e5b767269a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-577j2"
Jul 26 17:49:39.334739 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334691    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h554k\" (UniqueName: \"kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-kube-api-access-h554k\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.334739 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334710    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/801b12f8-cc05-4646-99a0-304297466b26-trusted-ca\") pod \"console-operator-584dcb6b54-q8vbm\" (UID: \"801b12f8-cc05-4646-99a0-304297466b26\") " pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm"
Jul 26 17:49:39.334739 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334724    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f532906-dddd-459d-b8da-3074eab679b1-trusted-ca\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.334933 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334741    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp7lg\" (UniqueName: \"kubernetes.io/projected/581ed0b5-91f3-4930-adfe-379ab913be79-kube-api-access-bp7lg\") pod \"cluster-samples-operator-6748489656-qdq4z\" (UID: \"581ed0b5-91f3-4930-adfe-379ab913be79\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qdq4z"
Jul 26 17:49:39.334933 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334762    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/801b12f8-cc05-4646-99a0-304297466b26-config\") pod \"console-operator-584dcb6b54-q8vbm\" (UID: \"801b12f8-cc05-4646-99a0-304297466b26\") " pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm"
Jul 26 17:49:39.334933 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.334779    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c07c88c4-6137-4a7e-b0ea-4e266eb50761-config-volume\") pod \"dns-default-26rxm\" (UID: \"c07c88c4-6137-4a7e-b0ea-4e266eb50761\") " pod="openshift-dns/dns-default-26rxm"
Jul 26 17:49:39.435971 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.435865    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lckds\" (UniqueName: \"kubernetes.io/projected/801b12f8-cc05-4646-99a0-304297466b26-kube-api-access-lckds\") pod \"console-operator-584dcb6b54-q8vbm\" (UID: \"801b12f8-cc05-4646-99a0-304297466b26\") " pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm"
Jul 26 17:49:39.435971 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.435903    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/40cb8009-e6c4-4fc2-a574-c4ecca296db4-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc\" (UID: \"40cb8009-e6c4-4fc2-a574-c4ecca296db4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc"
Jul 26 17:49:39.435971 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.435922    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/801b12f8-cc05-4646-99a0-304297466b26-serving-cert\") pod \"console-operator-584dcb6b54-q8vbm\" (UID: \"801b12f8-cc05-4646-99a0-304297466b26\") " pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm"
Jul 26 17:49:39.435971 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.435954    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-registry-tls\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.436342 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.435997    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f532906-dddd-459d-b8da-3074eab679b1-ca-trust-extracted\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.436342 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:39.436022    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 26 17:49:39.436342 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:39.436032    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f9b95bd64-5jvnp: secret "image-registry-tls" not found
Jul 26 17:49:39.436342 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436031    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3321b850-63be-4345-ac65-afaee093369a-cert\") pod \"ingress-canary-c9hsm\" (UID: \"3321b850-63be-4345-ac65-afaee093369a\") " pod="openshift-ingress-canary/ingress-canary-c9hsm"
Jul 26 17:49:39.436342 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436051    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-475zf\" (UniqueName: \"kubernetes.io/projected/c07c88c4-6137-4a7e-b0ea-4e266eb50761-kube-api-access-475zf\") pod \"dns-default-26rxm\" (UID: \"c07c88c4-6137-4a7e-b0ea-4e266eb50761\") " pod="openshift-dns/dns-default-26rxm"
Jul 26 17:49:39.436342 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:39.436094    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-registry-tls podName:0f532906-dddd-459d-b8da-3074eab679b1 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:39.9360756 +0000 UTC m=+36.588266437 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-registry-tls") pod "image-registry-f9b95bd64-5jvnp" (UID: "0f532906-dddd-459d-b8da-3074eab679b1") : secret "image-registry-tls" not found
Jul 26 17:49:39.436342 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436129    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v89p\" (UniqueName: \"kubernetes.io/projected/51d5a4bf-5978-4bb0-81e8-3004f8519805-kube-api-access-6v89p\") pod \"volume-data-source-validator-549c6d54cf-hsdvj\" (UID: \"51d5a4bf-5978-4bb0-81e8-3004f8519805\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hsdvj"
Jul 26 17:49:39.436342 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436159    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/40cb8009-e6c4-4fc2-a574-c4ecca296db4-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc\" (UID: \"40cb8009-e6c4-4fc2-a574-c4ecca296db4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc"
Jul 26 17:49:39.436342 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436182    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c07c88c4-6137-4a7e-b0ea-4e266eb50761-metrics-tls\") pod \"dns-default-26rxm\" (UID: \"c07c88c4-6137-4a7e-b0ea-4e266eb50761\") " pod="openshift-dns/dns-default-26rxm"
Jul 26 17:49:39.436342 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436208    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-bound-sa-token\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.436342 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436236    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f532906-dddd-459d-b8da-3074eab679b1-registry-certificates\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.436342 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436304    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e6cff45e-fc7b-46bd-a628-a5e5b767269a-serving-cert\") pod \"service-ca-operator-5d6c75f84f-577j2\" (UID: \"e6cff45e-fc7b-46bd-a628-a5e5b767269a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-577j2"
Jul 26 17:49:39.436342 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436338    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82q2n\" (UniqueName: \"kubernetes.io/projected/3321b850-63be-4345-ac65-afaee093369a-kube-api-access-82q2n\") pod \"ingress-canary-c9hsm\" (UID: \"3321b850-63be-4345-ac65-afaee093369a\") " pod="openshift-ingress-canary/ingress-canary-c9hsm"
Jul 26 17:49:39.436937 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436364    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e6cff45e-fc7b-46bd-a628-a5e5b767269a-config\") pod \"service-ca-operator-5d6c75f84f-577j2\" (UID: \"e6cff45e-fc7b-46bd-a628-a5e5b767269a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-577j2"
Jul 26 17:49:39.436937 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436388    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h554k\" (UniqueName: \"kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-kube-api-access-h554k\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.436937 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436423    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/801b12f8-cc05-4646-99a0-304297466b26-trusted-ca\") pod \"console-operator-584dcb6b54-q8vbm\" (UID: \"801b12f8-cc05-4646-99a0-304297466b26\") " pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm"
Jul 26 17:49:39.436937 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436447    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f532906-dddd-459d-b8da-3074eab679b1-trusted-ca\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.436937 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436467    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f532906-dddd-459d-b8da-3074eab679b1-ca-trust-extracted\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.436937 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436473    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp7lg\" (UniqueName: \"kubernetes.io/projected/581ed0b5-91f3-4930-adfe-379ab913be79-kube-api-access-bp7lg\") pod \"cluster-samples-operator-6748489656-qdq4z\" (UID: \"581ed0b5-91f3-4930-adfe-379ab913be79\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qdq4z"
Jul 26 17:49:39.436937 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436530    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/801b12f8-cc05-4646-99a0-304297466b26-config\") pod \"console-operator-584dcb6b54-q8vbm\" (UID: \"801b12f8-cc05-4646-99a0-304297466b26\") " pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm"
Jul 26 17:49:39.436937 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436561    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c07c88c4-6137-4a7e-b0ea-4e266eb50761-config-volume\") pod \"dns-default-26rxm\" (UID: \"c07c88c4-6137-4a7e-b0ea-4e266eb50761\") " pod="openshift-dns/dns-default-26rxm"
Jul 26 17:49:39.436937 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436563    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/40cb8009-e6c4-4fc2-a574-c4ecca296db4-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc\" (UID: \"40cb8009-e6c4-4fc2-a574-c4ecca296db4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc"
Jul 26 17:49:39.436937 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436593    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5szt\" (UniqueName: \"kubernetes.io/projected/5b18e275-81da-4553-921e-b95957e34eb6-kube-api-access-h5szt\") pod \"network-check-source-7d88cb55d8-z2knl\" (UID: \"5b18e275-81da-4553-921e-b95957e34eb6\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-z2knl"
Jul 26 17:49:39.436937 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436626    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f532906-dddd-459d-b8da-3074eab679b1-installation-pull-secrets\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.436937 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436655    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/581ed0b5-91f3-4930-adfe-379ab913be79-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-qdq4z\" (UID: \"581ed0b5-91f3-4930-adfe-379ab913be79\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qdq4z"
Jul 26 17:49:39.436937 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436682    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ms2v\" (UniqueName: \"kubernetes.io/projected/40cb8009-e6c4-4fc2-a574-c4ecca296db4-kube-api-access-7ms2v\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc\" (UID: \"40cb8009-e6c4-4fc2-a574-c4ecca296db4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc"
Jul 26 17:49:39.436937 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436727    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwr9f\" (UniqueName: \"kubernetes.io/projected/e6cff45e-fc7b-46bd-a628-a5e5b767269a-kube-api-access-wwr9f\") pod \"service-ca-operator-5d6c75f84f-577j2\" (UID: \"e6cff45e-fc7b-46bd-a628-a5e5b767269a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-577j2"
Jul 26 17:49:39.436937 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436753    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f532906-dddd-459d-b8da-3074eab679b1-image-registry-private-configuration\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.437643 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436777    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c07c88c4-6137-4a7e-b0ea-4e266eb50761-tmp-dir\") pod \"dns-default-26rxm\" (UID: \"c07c88c4-6137-4a7e-b0ea-4e266eb50761\") " pod="openshift-dns/dns-default-26rxm"
Jul 26 17:49:39.437643 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.436840    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f532906-dddd-459d-b8da-3074eab679b1-registry-certificates\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.437643 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.437127    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e6cff45e-fc7b-46bd-a628-a5e5b767269a-config\") pod \"service-ca-operator-5d6c75f84f-577j2\" (UID: \"e6cff45e-fc7b-46bd-a628-a5e5b767269a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-577j2"
Jul 26 17:49:39.437643 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:39.437148    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 26 17:49:39.437643 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:39.437195    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c07c88c4-6137-4a7e-b0ea-4e266eb50761-metrics-tls podName:c07c88c4-6137-4a7e-b0ea-4e266eb50761 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:39.93717943 +0000 UTC m=+36.589370266 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c07c88c4-6137-4a7e-b0ea-4e266eb50761-metrics-tls") pod "dns-default-26rxm" (UID: "c07c88c4-6137-4a7e-b0ea-4e266eb50761") : secret "dns-default-metrics-tls" not found
Jul 26 17:49:39.437643 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.437205    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/801b12f8-cc05-4646-99a0-304297466b26-config\") pod \"console-operator-584dcb6b54-q8vbm\" (UID: \"801b12f8-cc05-4646-99a0-304297466b26\") " pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm"
Jul 26 17:49:39.437643 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.437471    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c07c88c4-6137-4a7e-b0ea-4e266eb50761-tmp-dir\") pod \"dns-default-26rxm\" (UID: \"c07c88c4-6137-4a7e-b0ea-4e266eb50761\") " pod="openshift-dns/dns-default-26rxm"
Jul 26 17:49:39.437643 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.437640    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/801b12f8-cc05-4646-99a0-304297466b26-trusted-ca\") pod \"console-operator-584dcb6b54-q8vbm\" (UID: \"801b12f8-cc05-4646-99a0-304297466b26\") " pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm"
Jul 26 17:49:39.438491 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:39.438103    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 26 17:49:39.438491 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.438333    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f532906-dddd-459d-b8da-3074eab679b1-trusted-ca\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.438491 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:39.438399    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/581ed0b5-91f3-4930-adfe-379ab913be79-samples-operator-tls podName:581ed0b5-91f3-4930-adfe-379ab913be79 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:39.938383112 +0000 UTC m=+36.590573963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/581ed0b5-91f3-4930-adfe-379ab913be79-samples-operator-tls") pod "cluster-samples-operator-6748489656-qdq4z" (UID: "581ed0b5-91f3-4930-adfe-379ab913be79") : secret "samples-operator-tls" not found
Jul 26 17:49:39.439078 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.439053    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c07c88c4-6137-4a7e-b0ea-4e266eb50761-config-volume\") pod \"dns-default-26rxm\" (UID: \"c07c88c4-6137-4a7e-b0ea-4e266eb50761\") " pod="openshift-dns/dns-default-26rxm"
Jul 26 17:49:39.440923 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.440894    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/40cb8009-e6c4-4fc2-a574-c4ecca296db4-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc\" (UID: \"40cb8009-e6c4-4fc2-a574-c4ecca296db4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc"
Jul 26 17:49:39.441007 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.440900    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e6cff45e-fc7b-46bd-a628-a5e5b767269a-serving-cert\") pod \"service-ca-operator-5d6c75f84f-577j2\" (UID: \"e6cff45e-fc7b-46bd-a628-a5e5b767269a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-577j2"
Jul 26 17:49:39.441007 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.440970    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f532906-dddd-459d-b8da-3074eab679b1-installation-pull-secrets\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.441106 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.441067    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3321b850-63be-4345-ac65-afaee093369a-cert\") pod \"ingress-canary-c9hsm\" (UID: \"3321b850-63be-4345-ac65-afaee093369a\") " pod="openshift-ingress-canary/ingress-canary-c9hsm"
Jul 26 17:49:39.441275 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.441255    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/801b12f8-cc05-4646-99a0-304297466b26-serving-cert\") pod \"console-operator-584dcb6b54-q8vbm\" (UID: \"801b12f8-cc05-4646-99a0-304297466b26\") " pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm"
Jul 26 17:49:39.441348 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.441332    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f532906-dddd-459d-b8da-3074eab679b1-image-registry-private-configuration\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.445618 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.445568    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-475zf\" (UniqueName: \"kubernetes.io/projected/c07c88c4-6137-4a7e-b0ea-4e266eb50761-kube-api-access-475zf\") pod \"dns-default-26rxm\" (UID: \"c07c88c4-6137-4a7e-b0ea-4e266eb50761\") " pod="openshift-dns/dns-default-26rxm"
Jul 26 17:49:39.446001 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.445978    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82q2n\" (UniqueName: \"kubernetes.io/projected/3321b850-63be-4345-ac65-afaee093369a-kube-api-access-82q2n\") pod \"ingress-canary-c9hsm\" (UID: \"3321b850-63be-4345-ac65-afaee093369a\") " pod="openshift-ingress-canary/ingress-canary-c9hsm"
Jul 26 17:49:39.446088 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.446060    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-bound-sa-token\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.446509 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.446485    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp7lg\" (UniqueName: \"kubernetes.io/projected/581ed0b5-91f3-4930-adfe-379ab913be79-kube-api-access-bp7lg\") pod \"cluster-samples-operator-6748489656-qdq4z\" (UID: \"581ed0b5-91f3-4930-adfe-379ab913be79\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qdq4z"
Jul 26 17:49:39.446978 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.446959    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ms2v\" (UniqueName: \"kubernetes.io/projected/40cb8009-e6c4-4fc2-a574-c4ecca296db4-kube-api-access-7ms2v\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc\" (UID: \"40cb8009-e6c4-4fc2-a574-c4ecca296db4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc"
Jul 26 17:49:39.447082 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.447025    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lckds\" (UniqueName: \"kubernetes.io/projected/801b12f8-cc05-4646-99a0-304297466b26-kube-api-access-lckds\") pod \"console-operator-584dcb6b54-q8vbm\" (UID: \"801b12f8-cc05-4646-99a0-304297466b26\") " pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm"
Jul 26 17:49:39.447082 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.447037    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwr9f\" (UniqueName: \"kubernetes.io/projected/e6cff45e-fc7b-46bd-a628-a5e5b767269a-kube-api-access-wwr9f\") pod \"service-ca-operator-5d6c75f84f-577j2\" (UID: \"e6cff45e-fc7b-46bd-a628-a5e5b767269a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-577j2"
Jul 26 17:49:39.447965 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.447935    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h554k\" (UniqueName: \"kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-kube-api-access-h554k\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.448288 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.448268    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v89p\" (UniqueName: \"kubernetes.io/projected/51d5a4bf-5978-4bb0-81e8-3004f8519805-kube-api-access-6v89p\") pod \"volume-data-source-validator-549c6d54cf-hsdvj\" (UID: \"51d5a4bf-5978-4bb0-81e8-3004f8519805\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hsdvj"
Jul 26 17:49:39.448641 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.448622    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5szt\" (UniqueName: \"kubernetes.io/projected/5b18e275-81da-4553-921e-b95957e34eb6-kube-api-access-h5szt\") pod \"network-check-source-7d88cb55d8-z2knl\" (UID: \"5b18e275-81da-4553-921e-b95957e34eb6\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-z2knl"
Jul 26 17:49:39.510480 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.510446    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-m4lr7"]
Jul 26 17:49:39.515138 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.515102    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-m4lr7"
Jul 26 17:49:39.517557 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.517539    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 26 17:49:39.517658 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.517564    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 26 17:49:39.517658 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.517564    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-6bx4t\""
Jul 26 17:49:39.524051 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.524031    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-m4lr7"]
Jul 26 17:49:39.530410 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.530394    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hsdvj"
Jul 26 17:49:39.546412 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.546393    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm"
Jul 26 17:49:39.553868 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.553795    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-577j2"
Jul 26 17:49:39.559307 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.559286    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc"
Jul 26 17:49:39.565387 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.565230    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-z2knl"
Jul 26 17:49:39.572890 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.572865    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c9hsm"
Jul 26 17:49:39.638951 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.638704    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/df483052-a7ce-455d-b77b-471ef05da766-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-m4lr7\" (UID: \"df483052-a7ce-455d-b77b-471ef05da766\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-m4lr7"
Jul 26 17:49:39.639092 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.638994    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/df483052-a7ce-455d-b77b-471ef05da766-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-m4lr7\" (UID: \"df483052-a7ce-455d-b77b-471ef05da766\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-m4lr7"
Jul 26 17:49:39.741455 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.740165    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/df483052-a7ce-455d-b77b-471ef05da766-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-m4lr7\" (UID: \"df483052-a7ce-455d-b77b-471ef05da766\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-m4lr7"
Jul 26 17:49:39.741455 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.740242    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/df483052-a7ce-455d-b77b-471ef05da766-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-m4lr7\" (UID: \"df483052-a7ce-455d-b77b-471ef05da766\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-m4lr7"
Jul 26 17:49:39.741455 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:39.740707    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 26 17:49:39.741455 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:39.740782    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df483052-a7ce-455d-b77b-471ef05da766-networking-console-plugin-cert podName:df483052-a7ce-455d-b77b-471ef05da766 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:40.240757814 +0000 UTC m=+36.892948660 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/df483052-a7ce-455d-b77b-471ef05da766-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-m4lr7" (UID: "df483052-a7ce-455d-b77b-471ef05da766") : secret "networking-console-plugin-cert" not found
Jul 26 17:49:39.741455 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.741096    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/df483052-a7ce-455d-b77b-471ef05da766-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-m4lr7\" (UID: \"df483052-a7ce-455d-b77b-471ef05da766\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-m4lr7"
Jul 26 17:49:39.755573 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.755546    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hsdvj"]
Jul 26 17:49:39.761185 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.761104    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-q8vbm"]
Jul 26 17:49:39.762429 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:39.762324    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51d5a4bf_5978_4bb0_81e8_3004f8519805.slice/crio-5c718ffc0833784a7170257639a83786aecb52c9d76c18aeed2346fb34c0e926 WatchSource:0}: Error finding container 5c718ffc0833784a7170257639a83786aecb52c9d76c18aeed2346fb34c0e926: Status 404 returned error can't find the container with id 5c718ffc0833784a7170257639a83786aecb52c9d76c18aeed2346fb34c0e926
Jul 26 17:49:39.766840 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:39.766821    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod801b12f8_cc05_4646_99a0_304297466b26.slice/crio-03c29823d3f5f6d7ae452fd4d7d45dd563739bc96725b4018591dc14e6c1eef4 WatchSource:0}: Error finding container 03c29823d3f5f6d7ae452fd4d7d45dd563739bc96725b4018591dc14e6c1eef4: Status 404 returned error can't find the container with id 03c29823d3f5f6d7ae452fd4d7d45dd563739bc96725b4018591dc14e6c1eef4
Jul 26 17:49:39.777290 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.777257    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-577j2"]
Jul 26 17:49:39.780534 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:39.780506    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6cff45e_fc7b_46bd_a628_a5e5b767269a.slice/crio-f108b2eb560a361eda2f420418cc6dc0f663926a21d5b87b1b68457cf26c3982 WatchSource:0}: Error finding container f108b2eb560a361eda2f420418cc6dc0f663926a21d5b87b1b68457cf26c3982: Status 404 returned error can't find the container with id f108b2eb560a361eda2f420418cc6dc0f663926a21d5b87b1b68457cf26c3982
Jul 26 17:49:39.789310 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.789288    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc"]
Jul 26 17:49:39.792814 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:39.792787    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40cb8009_e6c4_4fc2_a574_c4ecca296db4.slice/crio-6ad2816eeb03dc6af8dc83df61a31404a4b7684db2b1ff81b6a80e57a942df03 WatchSource:0}: Error finding container 6ad2816eeb03dc6af8dc83df61a31404a4b7684db2b1ff81b6a80e57a942df03: Status 404 returned error can't find the container with id 6ad2816eeb03dc6af8dc83df61a31404a4b7684db2b1ff81b6a80e57a942df03
Jul 26 17:49:39.797639 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.797616    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-z2knl"]
Jul 26 17:49:39.800228 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.800205    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-c9hsm"]
Jul 26 17:49:39.800551 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:39.800524    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b18e275_81da_4553_921e_b95957e34eb6.slice/crio-f94438f340a4702da554d0f831060fc8279f2050943ff3312dddf914ee2fe948 WatchSource:0}: Error finding container f94438f340a4702da554d0f831060fc8279f2050943ff3312dddf914ee2fe948: Status 404 returned error can't find the container with id f94438f340a4702da554d0f831060fc8279f2050943ff3312dddf914ee2fe948
Jul 26 17:49:39.803021 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:49:39.802997    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3321b850_63be_4345_ac65_afaee093369a.slice/crio-94d2c223880526886544dd7ba4621a71ed1e85a98a8e1193519c1dadbd871e6b WatchSource:0}: Error finding container 94d2c223880526886544dd7ba4621a71ed1e85a98a8e1193519c1dadbd871e6b: Status 404 returned error can't find the container with id 94d2c223880526886544dd7ba4621a71ed1e85a98a8e1193519c1dadbd871e6b
Jul 26 17:49:39.832348 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.832322    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:49:39.833047 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.832322    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:49:39.836846 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.836821    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nb78b\""
Jul 26 17:49:39.836846 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.836833    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 26 17:49:39.836846 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.836848    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zs84s\""
Jul 26 17:49:39.941478 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.941441    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-registry-tls\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:39.941625 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.941491    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c07c88c4-6137-4a7e-b0ea-4e266eb50761-metrics-tls\") pod \"dns-default-26rxm\" (UID: \"c07c88c4-6137-4a7e-b0ea-4e266eb50761\") " pod="openshift-dns/dns-default-26rxm"
Jul 26 17:49:39.941625 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:39.941584    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 26 17:49:39.941625 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:39.941600    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 26 17:49:39.941625 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:39.941623    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f9b95bd64-5jvnp: secret "image-registry-tls" not found
Jul 26 17:49:39.941820 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:39.941631    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c07c88c4-6137-4a7e-b0ea-4e266eb50761-metrics-tls podName:c07c88c4-6137-4a7e-b0ea-4e266eb50761 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:40.941618348 +0000 UTC m=+37.593809184 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c07c88c4-6137-4a7e-b0ea-4e266eb50761-metrics-tls") pod "dns-default-26rxm" (UID: "c07c88c4-6137-4a7e-b0ea-4e266eb50761") : secret "dns-default-metrics-tls" not found
Jul 26 17:49:39.941820 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:39.941645    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/581ed0b5-91f3-4930-adfe-379ab913be79-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-qdq4z\" (UID: \"581ed0b5-91f3-4930-adfe-379ab913be79\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qdq4z"
Jul 26 17:49:39.941820 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:39.941677    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-registry-tls podName:0f532906-dddd-459d-b8da-3074eab679b1 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:40.94166049 +0000 UTC m=+37.593851345 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-registry-tls") pod "image-registry-f9b95bd64-5jvnp" (UID: "0f532906-dddd-459d-b8da-3074eab679b1") : secret "image-registry-tls" not found
Jul 26 17:49:39.941820 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:39.941698    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 26 17:49:39.941820 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:39.941733    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/581ed0b5-91f3-4930-adfe-379ab913be79-samples-operator-tls podName:581ed0b5-91f3-4930-adfe-379ab913be79 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:40.941722352 +0000 UTC m=+37.593913211 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/581ed0b5-91f3-4930-adfe-379ab913be79-samples-operator-tls") pod "cluster-samples-operator-6748489656-qdq4z" (UID: "581ed0b5-91f3-4930-adfe-379ab913be79") : secret "samples-operator-tls" not found
Jul 26 17:49:40.069522 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:40.069482    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-z2knl" event={"ID":"5b18e275-81da-4553-921e-b95957e34eb6","Type":"ContainerStarted","Data":"f94438f340a4702da554d0f831060fc8279f2050943ff3312dddf914ee2fe948"}
Jul 26 17:49:40.070490 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:40.070459    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm" event={"ID":"801b12f8-cc05-4646-99a0-304297466b26","Type":"ContainerStarted","Data":"03c29823d3f5f6d7ae452fd4d7d45dd563739bc96725b4018591dc14e6c1eef4"}
Jul 26 17:49:40.071504 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:40.071484    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-577j2" event={"ID":"e6cff45e-fc7b-46bd-a628-a5e5b767269a","Type":"ContainerStarted","Data":"f108b2eb560a361eda2f420418cc6dc0f663926a21d5b87b1b68457cf26c3982"}
Jul 26 17:49:40.072470 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:40.072450    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc" event={"ID":"40cb8009-e6c4-4fc2-a574-c4ecca296db4","Type":"ContainerStarted","Data":"6ad2816eeb03dc6af8dc83df61a31404a4b7684db2b1ff81b6a80e57a942df03"}
Jul 26 17:49:40.073548 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:40.073527    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hsdvj" event={"ID":"51d5a4bf-5978-4bb0-81e8-3004f8519805","Type":"ContainerStarted","Data":"5c718ffc0833784a7170257639a83786aecb52c9d76c18aeed2346fb34c0e926"}
Jul 26 17:49:40.076205 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:40.076179    2570 generic.go:356] "Generic (PLEG): container finished" podID="a337e0eb-4d13-4df5-bfe7-186de89d4dcc" containerID="e8618b9d0ba888c35ed8c92ab09d3deecf7630e7b7688e2e3997ea8d84fccbd4" exitCode=0
Jul 26 17:49:40.076286 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:40.076205    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4zzb" event={"ID":"a337e0eb-4d13-4df5-bfe7-186de89d4dcc","Type":"ContainerDied","Data":"e8618b9d0ba888c35ed8c92ab09d3deecf7630e7b7688e2e3997ea8d84fccbd4"}
Jul 26 17:49:40.077369 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:40.077345    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c9hsm" event={"ID":"3321b850-63be-4345-ac65-afaee093369a","Type":"ContainerStarted","Data":"94d2c223880526886544dd7ba4621a71ed1e85a98a8e1193519c1dadbd871e6b"}
Jul 26 17:49:40.245341 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:40.245305    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/df483052-a7ce-455d-b77b-471ef05da766-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-m4lr7\" (UID: \"df483052-a7ce-455d-b77b-471ef05da766\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-m4lr7"
Jul 26 17:49:40.245494 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:40.245474    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 26 17:49:40.245561 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:40.245552    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df483052-a7ce-455d-b77b-471ef05da766-networking-console-plugin-cert podName:df483052-a7ce-455d-b77b-471ef05da766 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:41.245532511 +0000 UTC m=+37.897723367 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/df483052-a7ce-455d-b77b-471ef05da766-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-m4lr7" (UID: "df483052-a7ce-455d-b77b-471ef05da766") : secret "networking-console-plugin-cert" not found
Jul 26 17:49:40.833062 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:40.832386    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:49:40.837200 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:40.835753    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 26 17:49:40.953459 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:40.953415    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c07c88c4-6137-4a7e-b0ea-4e266eb50761-metrics-tls\") pod \"dns-default-26rxm\" (UID: \"c07c88c4-6137-4a7e-b0ea-4e266eb50761\") " pod="openshift-dns/dns-default-26rxm"
Jul 26 17:49:40.953623 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:40.953517    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/581ed0b5-91f3-4930-adfe-379ab913be79-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-qdq4z\" (UID: \"581ed0b5-91f3-4930-adfe-379ab913be79\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qdq4z"
Jul 26 17:49:40.953623 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:40.953607    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-registry-tls\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:40.953753 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:40.953735    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 26 17:49:40.953753 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:40.953749    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f9b95bd64-5jvnp: secret "image-registry-tls" not found
Jul 26 17:49:40.953842 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:40.953809    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-registry-tls podName:0f532906-dddd-459d-b8da-3074eab679b1 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:42.95379091 +0000 UTC m=+39.605981750 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-registry-tls") pod "image-registry-f9b95bd64-5jvnp" (UID: "0f532906-dddd-459d-b8da-3074eab679b1") : secret "image-registry-tls" not found
Jul 26 17:49:40.954255 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:40.954232    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 26 17:49:40.954351 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:40.954291    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c07c88c4-6137-4a7e-b0ea-4e266eb50761-metrics-tls podName:c07c88c4-6137-4a7e-b0ea-4e266eb50761 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:42.954275595 +0000 UTC m=+39.606466431 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c07c88c4-6137-4a7e-b0ea-4e266eb50761-metrics-tls") pod "dns-default-26rxm" (UID: "c07c88c4-6137-4a7e-b0ea-4e266eb50761") : secret "dns-default-metrics-tls" not found
Jul 26 17:49:40.954415 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:40.954355    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 26 17:49:40.954415 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:40.954388    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/581ed0b5-91f3-4930-adfe-379ab913be79-samples-operator-tls podName:581ed0b5-91f3-4930-adfe-379ab913be79 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:42.954377669 +0000 UTC m=+39.606568509 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/581ed0b5-91f3-4930-adfe-379ab913be79-samples-operator-tls") pod "cluster-samples-operator-6748489656-qdq4z" (UID: "581ed0b5-91f3-4930-adfe-379ab913be79") : secret "samples-operator-tls" not found
Jul 26 17:49:41.088175 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:41.088078    2570 generic.go:356] "Generic (PLEG): container finished" podID="a337e0eb-4d13-4df5-bfe7-186de89d4dcc" containerID="7e0cfd372c7bfea30482a72946f40255b920bf3bb0b7f128b457853d891d84a9" exitCode=0
Jul 26 17:49:41.088324 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:41.088182    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4zzb" event={"ID":"a337e0eb-4d13-4df5-bfe7-186de89d4dcc","Type":"ContainerDied","Data":"7e0cfd372c7bfea30482a72946f40255b920bf3bb0b7f128b457853d891d84a9"}
Jul 26 17:49:41.257387 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:41.257350    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/df483052-a7ce-455d-b77b-471ef05da766-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-m4lr7\" (UID: \"df483052-a7ce-455d-b77b-471ef05da766\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-m4lr7"
Jul 26 17:49:41.258333 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:41.258266    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 26 17:49:41.258463 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:41.258347    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df483052-a7ce-455d-b77b-471ef05da766-networking-console-plugin-cert podName:df483052-a7ce-455d-b77b-471ef05da766 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:43.258326291 +0000 UTC m=+39.910517142 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/df483052-a7ce-455d-b77b-471ef05da766-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-m4lr7" (UID: "df483052-a7ce-455d-b77b-471ef05da766") : secret "networking-console-plugin-cert" not found
Jul 26 17:49:42.095176 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:42.094842    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hsdvj" event={"ID":"51d5a4bf-5978-4bb0-81e8-3004f8519805","Type":"ContainerStarted","Data":"5da10f5836f4702fd0caa76b4f2046e7d63e5a4e8e76bfe04a381ad4061f05fd"}
Jul 26 17:49:42.102496 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:42.102449    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4zzb" event={"ID":"a337e0eb-4d13-4df5-bfe7-186de89d4dcc","Type":"ContainerStarted","Data":"f717d7929c208a3c258d722d86587b58a450f6b4166d73247a4c1b454a4c30d1"}
Jul 26 17:49:42.136387 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:42.135226    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-s4zzb" podStartSLOduration=9.29530503 podStartE2EDuration="38.135203141s" totalImagesPullingTime="28.839898111s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:49:04 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:42.134306541 +0000 UTC m=+38.786497391" watchObservedRunningTime="2026-07-26 17:49:42.135203141 +0000 UTC m=+38.787394002"
Jul 26 17:49:42.136387 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:42.135561    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hsdvj" podStartSLOduration=34.502062135 podStartE2EDuration="36.135547161s" totalImagesPullingTime="1.633485026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:42.111947316 +0000 UTC m=+38.764138193" watchObservedRunningTime="2026-07-26 17:49:42.135547161 +0000 UTC m=+38.787738021"
Jul 26 17:49:42.973557 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:42.973519    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c07c88c4-6137-4a7e-b0ea-4e266eb50761-metrics-tls\") pod \"dns-default-26rxm\" (UID: \"c07c88c4-6137-4a7e-b0ea-4e266eb50761\") " pod="openshift-dns/dns-default-26rxm"
Jul 26 17:49:42.973717 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:42.973605    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/581ed0b5-91f3-4930-adfe-379ab913be79-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-qdq4z\" (UID: \"581ed0b5-91f3-4930-adfe-379ab913be79\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qdq4z"
Jul 26 17:49:42.973717 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:42.973678    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-registry-tls\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:42.973717 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:42.973686    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 26 17:49:42.973886 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:42.973760    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c07c88c4-6137-4a7e-b0ea-4e266eb50761-metrics-tls podName:c07c88c4-6137-4a7e-b0ea-4e266eb50761 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:46.973737925 +0000 UTC m=+43.625928762 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c07c88c4-6137-4a7e-b0ea-4e266eb50761-metrics-tls") pod "dns-default-26rxm" (UID: "c07c88c4-6137-4a7e-b0ea-4e266eb50761") : secret "dns-default-metrics-tls" not found
Jul 26 17:49:42.973886 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:42.973758    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 26 17:49:42.973886 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:42.973789    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 26 17:49:42.973886 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:42.973802    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f9b95bd64-5jvnp: secret "image-registry-tls" not found
Jul 26 17:49:42.973886 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:42.973803    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/581ed0b5-91f3-4930-adfe-379ab913be79-samples-operator-tls podName:581ed0b5-91f3-4930-adfe-379ab913be79 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:46.973793288 +0000 UTC m=+43.625984148 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/581ed0b5-91f3-4930-adfe-379ab913be79-samples-operator-tls") pod "cluster-samples-operator-6748489656-qdq4z" (UID: "581ed0b5-91f3-4930-adfe-379ab913be79") : secret "samples-operator-tls" not found
Jul 26 17:49:42.973886 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:42.973868    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-registry-tls podName:0f532906-dddd-459d-b8da-3074eab679b1 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:46.973852022 +0000 UTC m=+43.626042865 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-registry-tls") pod "image-registry-f9b95bd64-5jvnp" (UID: "0f532906-dddd-459d-b8da-3074eab679b1") : secret "image-registry-tls" not found
Jul 26 17:49:43.276791 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:43.276754    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/df483052-a7ce-455d-b77b-471ef05da766-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-m4lr7\" (UID: \"df483052-a7ce-455d-b77b-471ef05da766\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-m4lr7"
Jul 26 17:49:43.277158 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:43.276933    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 26 17:49:43.277158 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:43.277006    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df483052-a7ce-455d-b77b-471ef05da766-networking-console-plugin-cert podName:df483052-a7ce-455d-b77b-471ef05da766 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:47.27698883 +0000 UTC m=+43.929179675 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/df483052-a7ce-455d-b77b-471ef05da766-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-m4lr7" (UID: "df483052-a7ce-455d-b77b-471ef05da766") : secret "networking-console-plugin-cert" not found
Jul 26 17:49:47.011525 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:47.011478    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/581ed0b5-91f3-4930-adfe-379ab913be79-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-qdq4z\" (UID: \"581ed0b5-91f3-4930-adfe-379ab913be79\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qdq4z"
Jul 26 17:49:47.011979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:47.011595    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-registry-tls\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:47.011979 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:47.011629    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 26 17:49:47.011979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:47.011646    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c07c88c4-6137-4a7e-b0ea-4e266eb50761-metrics-tls\") pod \"dns-default-26rxm\" (UID: \"c07c88c4-6137-4a7e-b0ea-4e266eb50761\") " pod="openshift-dns/dns-default-26rxm"
Jul 26 17:49:47.011979 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:47.011701    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/581ed0b5-91f3-4930-adfe-379ab913be79-samples-operator-tls podName:581ed0b5-91f3-4930-adfe-379ab913be79 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:55.011681667 +0000 UTC m=+51.663872519 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/581ed0b5-91f3-4930-adfe-379ab913be79-samples-operator-tls") pod "cluster-samples-operator-6748489656-qdq4z" (UID: "581ed0b5-91f3-4930-adfe-379ab913be79") : secret "samples-operator-tls" not found
Jul 26 17:49:47.011979 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:47.011756    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 26 17:49:47.011979 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:47.011774    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 26 17:49:47.011979 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:47.011777    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f9b95bd64-5jvnp: secret "image-registry-tls" not found
Jul 26 17:49:47.011979 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:47.011821    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c07c88c4-6137-4a7e-b0ea-4e266eb50761-metrics-tls podName:c07c88c4-6137-4a7e-b0ea-4e266eb50761 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:55.011805742 +0000 UTC m=+51.663996585 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c07c88c4-6137-4a7e-b0ea-4e266eb50761-metrics-tls") pod "dns-default-26rxm" (UID: "c07c88c4-6137-4a7e-b0ea-4e266eb50761") : secret "dns-default-metrics-tls" not found
Jul 26 17:49:47.011979 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:47.011838    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-registry-tls podName:0f532906-dddd-459d-b8da-3074eab679b1 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:55.01182931 +0000 UTC m=+51.664020151 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-registry-tls") pod "image-registry-f9b95bd64-5jvnp" (UID: "0f532906-dddd-459d-b8da-3074eab679b1") : secret "image-registry-tls" not found
Jul 26 17:49:47.123124 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:47.123066    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c9hsm" event={"ID":"3321b850-63be-4345-ac65-afaee093369a","Type":"ContainerStarted","Data":"e600b62d1a57a4fee16da254fe2fe1ee745cff4b987114d3432484acb3db04e9"}
Jul 26 17:49:47.124636 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:47.124610    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-z2knl" event={"ID":"5b18e275-81da-4553-921e-b95957e34eb6","Type":"ContainerStarted","Data":"abeb5d011fb3affc136471f87566d3d223c94c1e786994448a5a75eef3de8440"}
Jul 26 17:49:47.126366 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:47.126330    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/0.log"
Jul 26 17:49:47.126473 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:47.126391    2570 generic.go:356] "Generic (PLEG): container finished" podID="801b12f8-cc05-4646-99a0-304297466b26" containerID="bffa94fb83e3466bd376c805480da81c049b0fcce57910f35024ffd6f0556db3" exitCode=255
Jul 26 17:49:47.126531 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:47.126470    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm" event={"ID":"801b12f8-cc05-4646-99a0-304297466b26","Type":"ContainerDied","Data":"bffa94fb83e3466bd376c805480da81c049b0fcce57910f35024ffd6f0556db3"}
Jul 26 17:49:47.126648 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:47.126632    2570 scope.go:117] "RemoveContainer" containerID="bffa94fb83e3466bd376c805480da81c049b0fcce57910f35024ffd6f0556db3"
Jul 26 17:49:47.129200 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:47.129177    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-577j2" event={"ID":"e6cff45e-fc7b-46bd-a628-a5e5b767269a","Type":"ContainerStarted","Data":"34a67c25c59a7e858bb96cf18cbd91f1fb220d1e159090d3692b9860ee17d3e8"}
Jul 26 17:49:47.130634 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:47.130616    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc" event={"ID":"40cb8009-e6c4-4fc2-a574-c4ecca296db4","Type":"ContainerStarted","Data":"4038cff4787926ca5b37785c35e778571ad77b8c8242763ac45074bb59bc293a"}
Jul 26 17:49:47.139298 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:47.139262    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-c9hsm" podStartSLOduration=1.394407113 podStartE2EDuration="8.139249336s" totalImagesPullingTime="6.744842223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:49:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:47.13883152 +0000 UTC m=+43.791022376" watchObservedRunningTime="2026-07-26 17:49:47.139249336 +0000 UTC m=+43.791440194"
Jul 26 17:49:47.186126 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:47.186062    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-577j2" podStartSLOduration=29.419041736 podStartE2EDuration="36.186041314s" totalImagesPullingTime="6.766999578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:49:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:47.184956953 +0000 UTC m=+43.837147808" watchObservedRunningTime="2026-07-26 17:49:47.186041314 +0000 UTC m=+43.838232173"
Jul 26 17:49:47.204474 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:47.204218    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc" podStartSLOduration=29.442715098 podStartE2EDuration="36.204199655s" totalImagesPullingTime="6.761484557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:49:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:47.20344448 +0000 UTC m=+43.855635338" watchObservedRunningTime="2026-07-26 17:49:47.204199655 +0000 UTC m=+43.856390512"
Jul 26 17:49:47.221377 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:47.220923    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-z2knl" podStartSLOduration=28.474337103 podStartE2EDuration="35.220905412s" totalImagesPullingTime="6.746568309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:49:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:47.220468698 +0000 UTC m=+43.872659571" watchObservedRunningTime="2026-07-26 17:49:47.220905412 +0000 UTC m=+43.873096270"
Jul 26 17:49:47.315361 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:47.315322    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/df483052-a7ce-455d-b77b-471ef05da766-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-m4lr7\" (UID: \"df483052-a7ce-455d-b77b-471ef05da766\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-m4lr7"
Jul 26 17:49:47.316093 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:47.315779    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 26 17:49:47.316093 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:47.315843    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df483052-a7ce-455d-b77b-471ef05da766-networking-console-plugin-cert podName:df483052-a7ce-455d-b77b-471ef05da766 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:55.315823111 +0000 UTC m=+51.968013952 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/df483052-a7ce-455d-b77b-471ef05da766-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-m4lr7" (UID: "df483052-a7ce-455d-b77b-471ef05da766") : secret "networking-console-plugin-cert" not found
Jul 26 17:49:48.141371 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:48.141345    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/1.log"
Jul 26 17:49:48.141867 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:48.141824    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/0.log"
Jul 26 17:49:48.141867 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:48.141856    2570 generic.go:356] "Generic (PLEG): container finished" podID="801b12f8-cc05-4646-99a0-304297466b26" containerID="688b53ee9e3fea7ce12a0c7b644bcf72a4fe26d4348740600158f0c34ddd1f37" exitCode=255
Jul 26 17:49:48.141999 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:48.141960    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm" event={"ID":"801b12f8-cc05-4646-99a0-304297466b26","Type":"ContainerDied","Data":"688b53ee9e3fea7ce12a0c7b644bcf72a4fe26d4348740600158f0c34ddd1f37"}
Jul 26 17:49:48.142050 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:48.142010    2570 scope.go:117] "RemoveContainer" containerID="bffa94fb83e3466bd376c805480da81c049b0fcce57910f35024ffd6f0556db3"
Jul 26 17:49:48.142242 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:48.142225    2570 scope.go:117] "RemoveContainer" containerID="688b53ee9e3fea7ce12a0c7b644bcf72a4fe26d4348740600158f0c34ddd1f37"
Jul 26 17:49:48.142421 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:48.142401    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-q8vbm_openshift-console-operator(801b12f8-cc05-4646-99a0-304297466b26)\"" pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm" podUID="801b12f8-cc05-4646-99a0-304297466b26"
Jul 26 17:49:49.146653 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:49.146622    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/1.log"
Jul 26 17:49:49.147049 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:49.146931    2570 scope.go:117] "RemoveContainer" containerID="688b53ee9e3fea7ce12a0c7b644bcf72a4fe26d4348740600158f0c34ddd1f37"
Jul 26 17:49:49.147091 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:49.147065    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-q8vbm_openshift-console-operator(801b12f8-cc05-4646-99a0-304297466b26)\"" pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm" podUID="801b12f8-cc05-4646-99a0-304297466b26"
Jul 26 17:49:49.547365 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:49.547334    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm"
Jul 26 17:49:49.547528 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:49.547393    2570 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm"
Jul 26 17:49:50.150408 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:50.150379    2570 scope.go:117] "RemoveContainer" containerID="688b53ee9e3fea7ce12a0c7b644bcf72a4fe26d4348740600158f0c34ddd1f37"
Jul 26 17:49:50.150773 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:50.150521    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-q8vbm_openshift-console-operator(801b12f8-cc05-4646-99a0-304297466b26)\"" pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm" podUID="801b12f8-cc05-4646-99a0-304297466b26"
Jul 26 17:49:55.078993 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:55.078957    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c07c88c4-6137-4a7e-b0ea-4e266eb50761-metrics-tls\") pod \"dns-default-26rxm\" (UID: \"c07c88c4-6137-4a7e-b0ea-4e266eb50761\") " pod="openshift-dns/dns-default-26rxm"
Jul 26 17:49:55.079398 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:55.079014    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/581ed0b5-91f3-4930-adfe-379ab913be79-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-qdq4z\" (UID: \"581ed0b5-91f3-4930-adfe-379ab913be79\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qdq4z"
Jul 26 17:49:55.079398 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:55.079096    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-registry-tls\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:49:55.079398 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:55.079211    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 26 17:49:55.079398 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:55.079223    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f9b95bd64-5jvnp: secret "image-registry-tls" not found
Jul 26 17:49:55.079398 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:55.079259    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 26 17:49:55.079398 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:55.079275    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-registry-tls podName:0f532906-dddd-459d-b8da-3074eab679b1 nodeName:}" failed. No retries permitted until 2026-07-26 17:50:11.079257196 +0000 UTC m=+67.731448042 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-registry-tls") pod "image-registry-f9b95bd64-5jvnp" (UID: "0f532906-dddd-459d-b8da-3074eab679b1") : secret "image-registry-tls" not found
Jul 26 17:49:55.079398 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:55.079316    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/581ed0b5-91f3-4930-adfe-379ab913be79-samples-operator-tls podName:581ed0b5-91f3-4930-adfe-379ab913be79 nodeName:}" failed. No retries permitted until 2026-07-26 17:50:11.079301054 +0000 UTC m=+67.731491910 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/581ed0b5-91f3-4930-adfe-379ab913be79-samples-operator-tls") pod "cluster-samples-operator-6748489656-qdq4z" (UID: "581ed0b5-91f3-4930-adfe-379ab913be79") : secret "samples-operator-tls" not found
Jul 26 17:49:55.079398 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:55.079209    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 26 17:49:55.079398 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:55.079342    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c07c88c4-6137-4a7e-b0ea-4e266eb50761-metrics-tls podName:c07c88c4-6137-4a7e-b0ea-4e266eb50761 nodeName:}" failed. No retries permitted until 2026-07-26 17:50:11.079336868 +0000 UTC m=+67.731527704 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c07c88c4-6137-4a7e-b0ea-4e266eb50761-metrics-tls") pod "dns-default-26rxm" (UID: "c07c88c4-6137-4a7e-b0ea-4e266eb50761") : secret "dns-default-metrics-tls" not found
Jul 26 17:49:55.381913 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:49:55.381829    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/df483052-a7ce-455d-b77b-471ef05da766-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-m4lr7\" (UID: \"df483052-a7ce-455d-b77b-471ef05da766\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-m4lr7"
Jul 26 17:49:55.382085 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:55.381984    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 26 17:49:55.382085 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:49:55.382050    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df483052-a7ce-455d-b77b-471ef05da766-networking-console-plugin-cert podName:df483052-a7ce-455d-b77b-471ef05da766 nodeName:}" failed. No retries permitted until 2026-07-26 17:50:11.382035444 +0000 UTC m=+68.034226285 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/df483052-a7ce-455d-b77b-471ef05da766-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-m4lr7" (UID: "df483052-a7ce-455d-b77b-471ef05da766") : secret "networking-console-plugin-cert" not found
Jul 26 17:50:04.070045 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:04.070015    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5zh25"
Jul 26 17:50:04.832743 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:04.832712    2570 scope.go:117] "RemoveContainer" containerID="688b53ee9e3fea7ce12a0c7b644bcf72a4fe26d4348740600158f0c34ddd1f37"
Jul 26 17:50:05.192590 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:05.192519    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 17:50:05.192980 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:05.192964    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/1.log"
Jul 26 17:50:05.193022 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:05.192995    2570 generic.go:356] "Generic (PLEG): container finished" podID="801b12f8-cc05-4646-99a0-304297466b26" containerID="a2b1d48397d8620a3f9fe18c5e1931212d62465328dc1f05e572ffd9ecb335b6" exitCode=255
Jul 26 17:50:05.193056 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:05.193030    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm" event={"ID":"801b12f8-cc05-4646-99a0-304297466b26","Type":"ContainerDied","Data":"a2b1d48397d8620a3f9fe18c5e1931212d62465328dc1f05e572ffd9ecb335b6"}
Jul 26 17:50:05.193088 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:05.193076    2570 scope.go:117] "RemoveContainer" containerID="688b53ee9e3fea7ce12a0c7b644bcf72a4fe26d4348740600158f0c34ddd1f37"
Jul 26 17:50:05.193396 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:05.193375    2570 scope.go:117] "RemoveContainer" containerID="a2b1d48397d8620a3f9fe18c5e1931212d62465328dc1f05e572ffd9ecb335b6"
Jul 26 17:50:05.193570 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:50:05.193545    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-q8vbm_openshift-console-operator(801b12f8-cc05-4646-99a0-304297466b26)\"" pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm" podUID="801b12f8-cc05-4646-99a0-304297466b26"
Jul 26 17:50:06.197909 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:06.197883    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 17:50:09.447101 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.447067    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-558cb5c8df-jwwxj"]
Jul 26 17:50:09.449578 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.449563    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-558cb5c8df-jwwxj"
Jul 26 17:50:09.454377 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.454355    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 26 17:50:09.454508 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.454383    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 26 17:50:09.455165 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.455145    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 26 17:50:09.455585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.455459    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 26 17:50:09.464556 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.464534    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-558cb5c8df-jwwxj"]
Jul 26 17:50:09.493952 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.493918    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6e8576d-3999-4c2b-b503-ee302a2650a7-tmp\") pod \"klusterlet-addon-workmgr-558cb5c8df-jwwxj\" (UID: \"b6e8576d-3999-4c2b-b503-ee302a2650a7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-558cb5c8df-jwwxj"
Jul 26 17:50:09.494078 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.494015    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b6e8576d-3999-4c2b-b503-ee302a2650a7-klusterlet-config\") pod \"klusterlet-addon-workmgr-558cb5c8df-jwwxj\" (UID: \"b6e8576d-3999-4c2b-b503-ee302a2650a7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-558cb5c8df-jwwxj"
Jul 26 17:50:09.494078 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.494048    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2rcp\" (UniqueName: \"kubernetes.io/projected/b6e8576d-3999-4c2b-b503-ee302a2650a7-kube-api-access-l2rcp\") pod \"klusterlet-addon-workmgr-558cb5c8df-jwwxj\" (UID: \"b6e8576d-3999-4c2b-b503-ee302a2650a7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-558cb5c8df-jwwxj"
Jul 26 17:50:09.524192 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.524159    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-65db774ff-4whvz"]
Jul 26 17:50:09.526034 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.526020    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-65db774ff-4whvz"
Jul 26 17:50:09.528478 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.528456    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 26 17:50:09.528584 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.528462    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 26 17:50:09.528646 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.528629    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 26 17:50:09.528827 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.528810    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 26 17:50:09.528901 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.528832    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-q49fk\""
Jul 26 17:50:09.531084 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.531064    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 26 17:50:09.531972 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.531952    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 26 17:50:09.545487 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.545468    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-65db774ff-4whvz"]
Jul 26 17:50:09.547465 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.547446    2570 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm"
Jul 26 17:50:09.547553 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.547514    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm"
Jul 26 17:50:09.547837 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.547823    2570 scope.go:117] "RemoveContainer" containerID="a2b1d48397d8620a3f9fe18c5e1931212d62465328dc1f05e572ffd9ecb335b6"
Jul 26 17:50:09.547976 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:50:09.547959    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-q8vbm_openshift-console-operator(801b12f8-cc05-4646-99a0-304297466b26)\"" pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm" podUID="801b12f8-cc05-4646-99a0-304297466b26"
Jul 26 17:50:09.595343 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.595313    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/759b21a0-1d29-4328-ad81-1e6c9f0c5b66-stats-auth\") pod \"router-default-65db774ff-4whvz\" (UID: \"759b21a0-1d29-4328-ad81-1e6c9f0c5b66\") " pod="openshift-ingress/router-default-65db774ff-4whvz"
Jul 26 17:50:09.595469 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.595348    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/759b21a0-1d29-4328-ad81-1e6c9f0c5b66-metrics-certs\") pod \"router-default-65db774ff-4whvz\" (UID: \"759b21a0-1d29-4328-ad81-1e6c9f0c5b66\") " pod="openshift-ingress/router-default-65db774ff-4whvz"
Jul 26 17:50:09.595469 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.595381    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6e8576d-3999-4c2b-b503-ee302a2650a7-tmp\") pod \"klusterlet-addon-workmgr-558cb5c8df-jwwxj\" (UID: \"b6e8576d-3999-4c2b-b503-ee302a2650a7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-558cb5c8df-jwwxj"
Jul 26 17:50:09.595469 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.595452    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnrcf\" (UniqueName: \"kubernetes.io/projected/759b21a0-1d29-4328-ad81-1e6c9f0c5b66-kube-api-access-rnrcf\") pod \"router-default-65db774ff-4whvz\" (UID: \"759b21a0-1d29-4328-ad81-1e6c9f0c5b66\") " pod="openshift-ingress/router-default-65db774ff-4whvz"
Jul 26 17:50:09.595629 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.595525    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af7dc810-9c37-4155-ab59-7c4196ffb50b-metrics-certs\") pod \"network-metrics-daemon-7mtnm\" (UID: \"af7dc810-9c37-4155-ab59-7c4196ffb50b\") " pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:50:09.595629 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.595557    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b6e8576d-3999-4c2b-b503-ee302a2650a7-klusterlet-config\") pod \"klusterlet-addon-workmgr-558cb5c8df-jwwxj\" (UID: \"b6e8576d-3999-4c2b-b503-ee302a2650a7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-558cb5c8df-jwwxj"
Jul 26 17:50:09.595724 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.595625    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/759b21a0-1d29-4328-ad81-1e6c9f0c5b66-service-ca-bundle\") pod \"router-default-65db774ff-4whvz\" (UID: \"759b21a0-1d29-4328-ad81-1e6c9f0c5b66\") " pod="openshift-ingress/router-default-65db774ff-4whvz"
Jul 26 17:50:09.595724 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.595663    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2rcp\" (UniqueName: \"kubernetes.io/projected/b6e8576d-3999-4c2b-b503-ee302a2650a7-kube-api-access-l2rcp\") pod \"klusterlet-addon-workmgr-558cb5c8df-jwwxj\" (UID: \"b6e8576d-3999-4c2b-b503-ee302a2650a7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-558cb5c8df-jwwxj"
Jul 26 17:50:09.595724 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.595716    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32cbd054-d7a7-4c00-be2e-012e16598a49-original-pull-secret\") pod \"global-pull-secret-syncer-96jft\" (UID: \"32cbd054-d7a7-4c00-be2e-012e16598a49\") " pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:50:09.595867 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.595749    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6e8576d-3999-4c2b-b503-ee302a2650a7-tmp\") pod \"klusterlet-addon-workmgr-558cb5c8df-jwwxj\" (UID: \"b6e8576d-3999-4c2b-b503-ee302a2650a7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-558cb5c8df-jwwxj"
Jul 26 17:50:09.595867 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.595824    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/759b21a0-1d29-4328-ad81-1e6c9f0c5b66-default-certificate\") pod \"router-default-65db774ff-4whvz\" (UID: \"759b21a0-1d29-4328-ad81-1e6c9f0c5b66\") " pod="openshift-ingress/router-default-65db774ff-4whvz"
Jul 26 17:50:09.597799 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.597783    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 26 17:50:09.597867 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.597852    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 26 17:50:09.599721 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.599697    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b6e8576d-3999-4c2b-b503-ee302a2650a7-klusterlet-config\") pod \"klusterlet-addon-workmgr-558cb5c8df-jwwxj\" (UID: \"b6e8576d-3999-4c2b-b503-ee302a2650a7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-558cb5c8df-jwwxj"
Jul 26 17:50:09.608196 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.608176    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af7dc810-9c37-4155-ab59-7c4196ffb50b-metrics-certs\") pod \"network-metrics-daemon-7mtnm\" (UID: \"af7dc810-9c37-4155-ab59-7c4196ffb50b\") " pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:50:09.608348 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.608332    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32cbd054-d7a7-4c00-be2e-012e16598a49-original-pull-secret\") pod \"global-pull-secret-syncer-96jft\" (UID: \"32cbd054-d7a7-4c00-be2e-012e16598a49\") " pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:50:09.612720 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.612696    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2rcp\" (UniqueName: \"kubernetes.io/projected/b6e8576d-3999-4c2b-b503-ee302a2650a7-kube-api-access-l2rcp\") pod \"klusterlet-addon-workmgr-558cb5c8df-jwwxj\" (UID: \"b6e8576d-3999-4c2b-b503-ee302a2650a7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-558cb5c8df-jwwxj"
Jul 26 17:50:09.646520 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.646500    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96jft"
Jul 26 17:50:09.696479 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.696446    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/759b21a0-1d29-4328-ad81-1e6c9f0c5b66-default-certificate\") pod \"router-default-65db774ff-4whvz\" (UID: \"759b21a0-1d29-4328-ad81-1e6c9f0c5b66\") " pod="openshift-ingress/router-default-65db774ff-4whvz"
Jul 26 17:50:09.696679 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.696651    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/759b21a0-1d29-4328-ad81-1e6c9f0c5b66-stats-auth\") pod \"router-default-65db774ff-4whvz\" (UID: \"759b21a0-1d29-4328-ad81-1e6c9f0c5b66\") " pod="openshift-ingress/router-default-65db774ff-4whvz"
Jul 26 17:50:09.696808 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.696696    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/759b21a0-1d29-4328-ad81-1e6c9f0c5b66-metrics-certs\") pod \"router-default-65db774ff-4whvz\" (UID: \"759b21a0-1d29-4328-ad81-1e6c9f0c5b66\") " pod="openshift-ingress/router-default-65db774ff-4whvz"
Jul 26 17:50:09.696808 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.696751    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnrcf\" (UniqueName: \"kubernetes.io/projected/759b21a0-1d29-4328-ad81-1e6c9f0c5b66-kube-api-access-rnrcf\") pod \"router-default-65db774ff-4whvz\" (UID: \"759b21a0-1d29-4328-ad81-1e6c9f0c5b66\") " pod="openshift-ingress/router-default-65db774ff-4whvz"
Jul 26 17:50:09.696931 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.696806    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jntqg\" (UniqueName: \"kubernetes.io/projected/76a782a1-4344-459a-8f5c-3e84c018f0f7-kube-api-access-jntqg\") pod \"network-check-target-h8sjb\" (UID: \"76a782a1-4344-459a-8f5c-3e84c018f0f7\") " pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:50:09.696931 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.696854    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/759b21a0-1d29-4328-ad81-1e6c9f0c5b66-service-ca-bundle\") pod \"router-default-65db774ff-4whvz\" (UID: \"759b21a0-1d29-4328-ad81-1e6c9f0c5b66\") " pod="openshift-ingress/router-default-65db774ff-4whvz"
Jul 26 17:50:09.697686 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.697608    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/759b21a0-1d29-4328-ad81-1e6c9f0c5b66-service-ca-bundle\") pod \"router-default-65db774ff-4whvz\" (UID: \"759b21a0-1d29-4328-ad81-1e6c9f0c5b66\") " pod="openshift-ingress/router-default-65db774ff-4whvz"
Jul 26 17:50:09.699366 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.699338    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/759b21a0-1d29-4328-ad81-1e6c9f0c5b66-default-certificate\") pod \"router-default-65db774ff-4whvz\" (UID: \"759b21a0-1d29-4328-ad81-1e6c9f0c5b66\") " pod="openshift-ingress/router-default-65db774ff-4whvz"
Jul 26 17:50:09.699461 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.699429    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jntqg\" (UniqueName: \"kubernetes.io/projected/76a782a1-4344-459a-8f5c-3e84c018f0f7-kube-api-access-jntqg\") pod \"network-check-target-h8sjb\" (UID: \"76a782a1-4344-459a-8f5c-3e84c018f0f7\") " pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:50:09.699557 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.699536    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/759b21a0-1d29-4328-ad81-1e6c9f0c5b66-stats-auth\") pod \"router-default-65db774ff-4whvz\" (UID: \"759b21a0-1d29-4328-ad81-1e6c9f0c5b66\") " pod="openshift-ingress/router-default-65db774ff-4whvz"
Jul 26 17:50:09.699607 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.699544    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/759b21a0-1d29-4328-ad81-1e6c9f0c5b66-metrics-certs\") pod \"router-default-65db774ff-4whvz\" (UID: \"759b21a0-1d29-4328-ad81-1e6c9f0c5b66\") " pod="openshift-ingress/router-default-65db774ff-4whvz"
Jul 26 17:50:09.705872 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.705845    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnrcf\" (UniqueName: \"kubernetes.io/projected/759b21a0-1d29-4328-ad81-1e6c9f0c5b66-kube-api-access-rnrcf\") pod \"router-default-65db774ff-4whvz\" (UID: \"759b21a0-1d29-4328-ad81-1e6c9f0c5b66\") " pod="openshift-ingress/router-default-65db774ff-4whvz"
Jul 26 17:50:09.757722 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.757693    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-558cb5c8df-jwwxj"
Jul 26 17:50:09.767009 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.766988    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-96jft"]
Jul 26 17:50:09.769837 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:50:09.769810    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32cbd054_d7a7_4c00_be2e_012e16598a49.slice/crio-552178f986cdd2175a4226117520b493d615bf52fce2103b60cb8b01632d6c87 WatchSource:0}: Error finding container 552178f986cdd2175a4226117520b493d615bf52fce2103b60cb8b01632d6c87: Status 404 returned error can't find the container with id 552178f986cdd2175a4226117520b493d615bf52fce2103b60cb8b01632d6c87
Jul 26 17:50:09.835280 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.835250    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-65db774ff-4whvz"
Jul 26 17:50:09.849302 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.848980    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nb78b\""
Jul 26 17:50:09.853693 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.853670    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zs84s\""
Jul 26 17:50:09.857698 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.857671    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:50:09.862041 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.862018    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mtnm"
Jul 26 17:50:09.885635 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.885587    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-558cb5c8df-jwwxj"]
Jul 26 17:50:09.889035 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:50:09.888636    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6e8576d_3999_4c2b_b503_ee302a2650a7.slice/crio-47be37df2b143752f9f3f03aba9815c5f1c1691c55120a3ebc243f41b2aaf48d WatchSource:0}: Error finding container 47be37df2b143752f9f3f03aba9815c5f1c1691c55120a3ebc243f41b2aaf48d: Status 404 returned error can't find the container with id 47be37df2b143752f9f3f03aba9815c5f1c1691c55120a3ebc243f41b2aaf48d
Jul 26 17:50:09.988771 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:09.988703    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-65db774ff-4whvz"]
Jul 26 17:50:09.991805 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:50:09.991780    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod759b21a0_1d29_4328_ad81_1e6c9f0c5b66.slice/crio-616f02719ee9da74ee85462184b5ef5aa7248c2eeb20efa25dae902a14166a0f WatchSource:0}: Error finding container 616f02719ee9da74ee85462184b5ef5aa7248c2eeb20efa25dae902a14166a0f: Status 404 returned error can't find the container with id 616f02719ee9da74ee85462184b5ef5aa7248c2eeb20efa25dae902a14166a0f
Jul 26 17:50:10.011232 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:10.011206    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7mtnm"]
Jul 26 17:50:10.022627 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:10.022603    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-h8sjb"]
Jul 26 17:50:10.024264 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:50:10.024239    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf7dc810_9c37_4155_ab59_7c4196ffb50b.slice/crio-2d3a19d40a5276a39b787b403a28971bf4f610304f1f2915b1495264d32e86e7 WatchSource:0}: Error finding container 2d3a19d40a5276a39b787b403a28971bf4f610304f1f2915b1495264d32e86e7: Status 404 returned error can't find the container with id 2d3a19d40a5276a39b787b403a28971bf4f610304f1f2915b1495264d32e86e7
Jul 26 17:50:10.025816 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:50:10.025795    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76a782a1_4344_459a_8f5c_3e84c018f0f7.slice/crio-0f5e376a345ed3aef33471490261e31e275109476ed31b5f670e68a9a94e0cb7 WatchSource:0}: Error finding container 0f5e376a345ed3aef33471490261e31e275109476ed31b5f670e68a9a94e0cb7: Status 404 returned error can't find the container with id 0f5e376a345ed3aef33471490261e31e275109476ed31b5f670e68a9a94e0cb7
Jul 26 17:50:10.211230 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:10.211140    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7mtnm" event={"ID":"af7dc810-9c37-4155-ab59-7c4196ffb50b","Type":"ContainerStarted","Data":"2d3a19d40a5276a39b787b403a28971bf4f610304f1f2915b1495264d32e86e7"}
Jul 26 17:50:10.212813 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:10.212784    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-65db774ff-4whvz" event={"ID":"759b21a0-1d29-4328-ad81-1e6c9f0c5b66","Type":"ContainerStarted","Data":"5b15cc070d0e790767c209ca329c9e44e6f43ae7d9b5baf636b5283dce88cc47"}
Jul 26 17:50:10.212943 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:10.212834    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-65db774ff-4whvz" event={"ID":"759b21a0-1d29-4328-ad81-1e6c9f0c5b66","Type":"ContainerStarted","Data":"616f02719ee9da74ee85462184b5ef5aa7248c2eeb20efa25dae902a14166a0f"}
Jul 26 17:50:10.214314 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:10.214292    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-h8sjb" event={"ID":"76a782a1-4344-459a-8f5c-3e84c018f0f7","Type":"ContainerStarted","Data":"a8f5ecff61859ccdd8b055111caadf7240b6502397df4ea6a84c8f50a7020976"}
Jul 26 17:50:10.214415 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:10.214336    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:50:10.214415 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:10.214355    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-h8sjb" event={"ID":"76a782a1-4344-459a-8f5c-3e84c018f0f7","Type":"ContainerStarted","Data":"0f5e376a345ed3aef33471490261e31e275109476ed31b5f670e68a9a94e0cb7"}
Jul 26 17:50:10.215498 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:10.215473    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-558cb5c8df-jwwxj" event={"ID":"b6e8576d-3999-4c2b-b503-ee302a2650a7","Type":"ContainerStarted","Data":"47be37df2b143752f9f3f03aba9815c5f1c1691c55120a3ebc243f41b2aaf48d"}
Jul 26 17:50:10.216561 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:10.216540    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-96jft" event={"ID":"32cbd054-d7a7-4c00-be2e-012e16598a49","Type":"ContainerStarted","Data":"552178f986cdd2175a4226117520b493d615bf52fce2103b60cb8b01632d6c87"}
Jul 26 17:50:10.236008 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:10.235959    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-65db774ff-4whvz" podStartSLOduration=1.235942397 podStartE2EDuration="1.235942397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:50:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:10.234254739 +0000 UTC m=+66.886445596" watchObservedRunningTime="2026-07-26 17:50:10.235942397 +0000 UTC m=+66.888133256"
Jul 26 17:50:10.250236 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:10.250185    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-h8sjb" podStartSLOduration=66.250169165 podStartE2EDuration="1m6.250169165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:49:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:10.249286465 +0000 UTC m=+66.901477323" watchObservedRunningTime="2026-07-26 17:50:10.250169165 +0000 UTC m=+66.902360022"
Jul 26 17:50:10.835497 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:10.835442    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-65db774ff-4whvz"
Jul 26 17:50:10.838540 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:10.838508    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-65db774ff-4whvz"
Jul 26 17:50:11.109876 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:11.109830    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/581ed0b5-91f3-4930-adfe-379ab913be79-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-qdq4z\" (UID: \"581ed0b5-91f3-4930-adfe-379ab913be79\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qdq4z"
Jul 26 17:50:11.110078 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:11.109918    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-registry-tls\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:50:11.110078 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:11.109980    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c07c88c4-6137-4a7e-b0ea-4e266eb50761-metrics-tls\") pod \"dns-default-26rxm\" (UID: \"c07c88c4-6137-4a7e-b0ea-4e266eb50761\") " pod="openshift-dns/dns-default-26rxm"
Jul 26 17:50:11.112924 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:11.112897    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c07c88c4-6137-4a7e-b0ea-4e266eb50761-metrics-tls\") pod \"dns-default-26rxm\" (UID: \"c07c88c4-6137-4a7e-b0ea-4e266eb50761\") " pod="openshift-dns/dns-default-26rxm"
Jul 26 17:50:11.113056 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:11.112957    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-registry-tls\") pod \"image-registry-f9b95bd64-5jvnp\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") " pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:50:11.113193 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:11.113149    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/581ed0b5-91f3-4930-adfe-379ab913be79-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-qdq4z\" (UID: \"581ed0b5-91f3-4930-adfe-379ab913be79\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qdq4z"
Jul 26 17:50:11.221005 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:11.220972    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-65db774ff-4whvz"
Jul 26 17:50:11.222225 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:11.222200    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-65db774ff-4whvz"
Jul 26 17:50:11.319189 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:11.319099    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2hr9p\""
Jul 26 17:50:11.326023 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:11.326001    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:50:11.338320 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:11.338300    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wkmvr\""
Jul 26 17:50:11.346712 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:11.346687    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qdq4z"
Jul 26 17:50:11.378399 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:11.378374    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jb7dx\""
Jul 26 17:50:11.387502 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:11.387472    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-26rxm"
Jul 26 17:50:11.414616 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:11.414096    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/df483052-a7ce-455d-b77b-471ef05da766-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-m4lr7\" (UID: \"df483052-a7ce-455d-b77b-471ef05da766\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-m4lr7"
Jul 26 17:50:11.417370 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:11.417257    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/df483052-a7ce-455d-b77b-471ef05da766-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-m4lr7\" (UID: \"df483052-a7ce-455d-b77b-471ef05da766\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-m4lr7"
Jul 26 17:50:11.499167 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:11.499098    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f9b95bd64-5jvnp"]
Jul 26 17:50:11.512054 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:11.511960    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qdq4z"]
Jul 26 17:50:11.570108 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:11.570075    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-26rxm"]
Jul 26 17:50:11.626436 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:11.626358    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-6bx4t\""
Jul 26 17:50:11.634452 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:11.634428    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-m4lr7"
Jul 26 17:50:11.668481 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:50:11.668429    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f532906_dddd_459d_b8da_3074eab679b1.slice/crio-3a8eeb1f39e760a4b284d34869c8f4fa1630478daf1b089ff02c449617852094 WatchSource:0}: Error finding container 3a8eeb1f39e760a4b284d34869c8f4fa1630478daf1b089ff02c449617852094: Status 404 returned error can't find the container with id 3a8eeb1f39e760a4b284d34869c8f4fa1630478daf1b089ff02c449617852094
Jul 26 17:50:11.670101 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:50:11.670072    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc07c88c4_6137_4a7e_b0ea_4e266eb50761.slice/crio-ac13c3f128886c6c3fc68ee3cf4d54e95a696a11eb7159c424300b9134f402cc WatchSource:0}: Error finding container ac13c3f128886c6c3fc68ee3cf4d54e95a696a11eb7159c424300b9134f402cc: Status 404 returned error can't find the container with id ac13c3f128886c6c3fc68ee3cf4d54e95a696a11eb7159c424300b9134f402cc
Jul 26 17:50:11.860287 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:11.860254    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-m4lr7"]
Jul 26 17:50:11.863713 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:50:11.863685    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf483052_a7ce_455d_b77b_471ef05da766.slice/crio-79a3a4bbbb5ff4f09383e0f66595733e68d55ed16a4878df52ae1b918ae423af WatchSource:0}: Error finding container 79a3a4bbbb5ff4f09383e0f66595733e68d55ed16a4878df52ae1b918ae423af: Status 404 returned error can't find the container with id 79a3a4bbbb5ff4f09383e0f66595733e68d55ed16a4878df52ae1b918ae423af
Jul 26 17:50:12.227836 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:12.227745    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7mtnm" event={"ID":"af7dc810-9c37-4155-ab59-7c4196ffb50b","Type":"ContainerStarted","Data":"f4e0d133e2d24b5de86cbecf612ffbafdd90834b09a6b3c687e3662cc89fd2f1"}
Jul 26 17:50:12.227836 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:12.227804    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7mtnm" event={"ID":"af7dc810-9c37-4155-ab59-7c4196ffb50b","Type":"ContainerStarted","Data":"3da91f0c654fc8c3ecd5e710ff7c72373e9c86817d49da255b213a1eb98a96ac"}
Jul 26 17:50:12.230127 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:12.230091    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-m4lr7" event={"ID":"df483052-a7ce-455d-b77b-471ef05da766","Type":"ContainerStarted","Data":"79a3a4bbbb5ff4f09383e0f66595733e68d55ed16a4878df52ae1b918ae423af"}
Jul 26 17:50:12.232043 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:12.231988    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qdq4z" event={"ID":"581ed0b5-91f3-4930-adfe-379ab913be79","Type":"ContainerStarted","Data":"6fd784a763c87ff5b9516503504e629f23386792d5b8d85401764954816f9431"}
Jul 26 17:50:12.235214 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:12.234719    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp" event={"ID":"0f532906-dddd-459d-b8da-3074eab679b1","Type":"ContainerStarted","Data":"a65e029c7dc115ee36358b94bd24d0fda180f19cd43a5bf8456ec7cd721ff6cc"}
Jul 26 17:50:12.235214 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:12.234760    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp" event={"ID":"0f532906-dddd-459d-b8da-3074eab679b1","Type":"ContainerStarted","Data":"3a8eeb1f39e760a4b284d34869c8f4fa1630478daf1b089ff02c449617852094"}
Jul 26 17:50:12.235371 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:12.235271    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:50:12.236697 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:12.236675    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-26rxm" event={"ID":"c07c88c4-6137-4a7e-b0ea-4e266eb50761","Type":"ContainerStarted","Data":"ac13c3f128886c6c3fc68ee3cf4d54e95a696a11eb7159c424300b9134f402cc"}
Jul 26 17:50:12.262983 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:12.262910    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7mtnm" podStartSLOduration=66.589151811 podStartE2EDuration="1m8.26289142s" totalImagesPullingTime="1.673739609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:49:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:12.244881447 +0000 UTC m=+68.897072306" watchObservedRunningTime="2026-07-26 17:50:12.26289142 +0000 UTC m=+68.915082289"
Jul 26 17:50:12.263163 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:12.263104    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp" podStartSLOduration=68.263076533 podStartE2EDuration="1m8.263076533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:49:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:12.262622561 +0000 UTC m=+68.914813443" watchObservedRunningTime="2026-07-26 17:50:12.263076533 +0000 UTC m=+68.915267391"
Jul 26 17:50:15.690315 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:15.690278    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kdznm"]
Jul 26 17:50:15.741191 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:15.741148    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kdznm"]
Jul 26 17:50:15.741381 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:15.741302    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:15.748423 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:15.748398    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 26 17:50:15.748553 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:15.748400    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 26 17:50:15.748553 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:15.748397    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 26 17:50:15.748553 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:15.748400    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 26 17:50:15.749221 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:15.749180    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dk9zk\""
Jul 26 17:50:15.853309 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:15.853274    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/28761b10-b1ee-441e-946c-bdd7ad7e9eed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:15.853309 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:15.853311    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjdpv\" (UniqueName: \"kubernetes.io/projected/28761b10-b1ee-441e-946c-bdd7ad7e9eed-kube-api-access-mjdpv\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:15.853528 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:15.853351    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/28761b10-b1ee-441e-946c-bdd7ad7e9eed-crio-socket\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:15.853528 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:15.853394    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/28761b10-b1ee-441e-946c-bdd7ad7e9eed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:15.853528 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:15.853444    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/28761b10-b1ee-441e-946c-bdd7ad7e9eed-data-volume\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:15.954537 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:15.954464    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/28761b10-b1ee-441e-946c-bdd7ad7e9eed-crio-socket\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:15.954537 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:15.954507    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/28761b10-b1ee-441e-946c-bdd7ad7e9eed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:15.954537 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:15.954525    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/28761b10-b1ee-441e-946c-bdd7ad7e9eed-data-volume\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:15.954789 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:15.954571    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/28761b10-b1ee-441e-946c-bdd7ad7e9eed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:15.954789 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:15.954593    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjdpv\" (UniqueName: \"kubernetes.io/projected/28761b10-b1ee-441e-946c-bdd7ad7e9eed-kube-api-access-mjdpv\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:15.954789 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:50:15.954597    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 26 17:50:15.954789 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:15.954601    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/28761b10-b1ee-441e-946c-bdd7ad7e9eed-crio-socket\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:15.954789 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:50:15.954682    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/28761b10-b1ee-441e-946c-bdd7ad7e9eed-kube-rbac-proxy-cm podName:28761b10-b1ee-441e-946c-bdd7ad7e9eed nodeName:}" failed. No retries permitted until 2026-07-26 17:50:16.454664001 +0000 UTC m=+73.106854857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/28761b10-b1ee-441e-946c-bdd7ad7e9eed-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kdznm" (UID: "28761b10-b1ee-441e-946c-bdd7ad7e9eed") : configmap "kube-rbac-proxy" not found
Jul 26 17:50:15.954789 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:50:15.954727    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 26 17:50:15.954995 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:50:15.954797    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28761b10-b1ee-441e-946c-bdd7ad7e9eed-insights-runtime-extractor-tls podName:28761b10-b1ee-441e-946c-bdd7ad7e9eed nodeName:}" failed. No retries permitted until 2026-07-26 17:50:16.454778841 +0000 UTC m=+73.106969694 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/28761b10-b1ee-441e-946c-bdd7ad7e9eed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kdznm" (UID: "28761b10-b1ee-441e-946c-bdd7ad7e9eed") : secret "insights-runtime-extractor-tls" not found
Jul 26 17:50:15.954995 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:15.954902    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/28761b10-b1ee-441e-946c-bdd7ad7e9eed-data-volume\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:15.964397 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:15.964379    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjdpv\" (UniqueName: \"kubernetes.io/projected/28761b10-b1ee-441e-946c-bdd7ad7e9eed-kube-api-access-mjdpv\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:16.459522 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:16.459190    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/28761b10-b1ee-441e-946c-bdd7ad7e9eed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:16.459522 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:16.459296    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/28761b10-b1ee-441e-946c-bdd7ad7e9eed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:16.459522 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:50:16.459300    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 26 17:50:16.459522 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:50:16.459367    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/28761b10-b1ee-441e-946c-bdd7ad7e9eed-kube-rbac-proxy-cm podName:28761b10-b1ee-441e-946c-bdd7ad7e9eed nodeName:}" failed. No retries permitted until 2026-07-26 17:50:17.459345495 +0000 UTC m=+74.111536336 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/28761b10-b1ee-441e-946c-bdd7ad7e9eed-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kdznm" (UID: "28761b10-b1ee-441e-946c-bdd7ad7e9eed") : configmap "kube-rbac-proxy" not found
Jul 26 17:50:16.459522 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:50:16.459442    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 26 17:50:16.459522 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:50:16.459495    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28761b10-b1ee-441e-946c-bdd7ad7e9eed-insights-runtime-extractor-tls podName:28761b10-b1ee-441e-946c-bdd7ad7e9eed nodeName:}" failed. No retries permitted until 2026-07-26 17:50:17.459479804 +0000 UTC m=+74.111670653 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/28761b10-b1ee-441e-946c-bdd7ad7e9eed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kdznm" (UID: "28761b10-b1ee-441e-946c-bdd7ad7e9eed") : secret "insights-runtime-extractor-tls" not found
Jul 26 17:50:17.256963 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:17.256923    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-m4lr7" event={"ID":"df483052-a7ce-455d-b77b-471ef05da766","Type":"ContainerStarted","Data":"65e4ec64d1d2870fccd235f81f45b8505c2ae7c320d972c30d2a115785a79d3d"}
Jul 26 17:50:17.259212 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:17.259176    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qdq4z" event={"ID":"581ed0b5-91f3-4930-adfe-379ab913be79","Type":"ContainerStarted","Data":"dd915c0d7b94bb9daa59f76d86d1c785dece3abb35dacb2e33b0aa28d437eedb"}
Jul 26 17:50:17.259339 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:17.259279    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qdq4z" event={"ID":"581ed0b5-91f3-4930-adfe-379ab913be79","Type":"ContainerStarted","Data":"30ef9f6a190ba931b4f47fcf99fd78e0e30073d0823cb5301da55b856e5a4e0e"}
Jul 26 17:50:17.260806 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:17.260777    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-96jft" event={"ID":"32cbd054-d7a7-4c00-be2e-012e16598a49","Type":"ContainerStarted","Data":"2e2a5f8de96d13711da85a6c7ea0f8ceb4fbc7018230d8923ef5e12fd41e1b91"}
Jul 26 17:50:17.262717 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:17.262692    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-26rxm" event={"ID":"c07c88c4-6137-4a7e-b0ea-4e266eb50761","Type":"ContainerStarted","Data":"e30ec832f766c16f4bdd09befb447458812e6cd11e3fc39a11cf6c57a2366ffd"}
Jul 26 17:50:17.262804 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:17.262741    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-26rxm" event={"ID":"c07c88c4-6137-4a7e-b0ea-4e266eb50761","Type":"ContainerStarted","Data":"82b5550ba275cd18f31e6f9a302fd78c44c84ad3d220b9da041a017f395033a3"}
Jul 26 17:50:17.262926 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:17.262912    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-26rxm"
Jul 26 17:50:17.274741 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:17.274697    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-m4lr7" podStartSLOduration=33.837660164 podStartE2EDuration="38.274684173s" totalImagesPullingTime="4.437024009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:49:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:17.273639623 +0000 UTC m=+73.925830483" watchObservedRunningTime="2026-07-26 17:50:17.274684173 +0000 UTC m=+73.926875048"
Jul 26 17:50:17.292794 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:17.291437    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-26rxm" podStartSLOduration=33.681149606 podStartE2EDuration="38.291418608s" totalImagesPullingTime="4.610269002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:49:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:17.290868441 +0000 UTC m=+73.943059300" watchObservedRunningTime="2026-07-26 17:50:17.291418608 +0000 UTC m=+73.943609466"
Jul 26 17:50:17.307233 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:17.307164    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-96jft" podStartSLOduration=66.775905176 podStartE2EDuration="1m13.30714878s" totalImagesPullingTime="6.531243604s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:49:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:17.305311841 +0000 UTC m=+73.957502700" watchObservedRunningTime="2026-07-26 17:50:17.30714878 +0000 UTC m=+73.959339639"
Jul 26 17:50:17.324362 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:17.323749    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qdq4z" podStartSLOduration=66.784085887 podStartE2EDuration="1m11.323729038s" totalImagesPullingTime="4.539643151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:17.322401593 +0000 UTC m=+73.974592454" watchObservedRunningTime="2026-07-26 17:50:17.323729038 +0000 UTC m=+73.975919897"
Jul 26 17:50:17.471699 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:17.471623    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/28761b10-b1ee-441e-946c-bdd7ad7e9eed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:17.471892 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:50:17.471789    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 26 17:50:17.471892 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:17.471800    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/28761b10-b1ee-441e-946c-bdd7ad7e9eed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:17.471892 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:50:17.471874    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28761b10-b1ee-441e-946c-bdd7ad7e9eed-insights-runtime-extractor-tls podName:28761b10-b1ee-441e-946c-bdd7ad7e9eed nodeName:}" failed. No retries permitted until 2026-07-26 17:50:19.471848465 +0000 UTC m=+76.124039318 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/28761b10-b1ee-441e-946c-bdd7ad7e9eed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kdznm" (UID: "28761b10-b1ee-441e-946c-bdd7ad7e9eed") : secret "insights-runtime-extractor-tls" not found
Jul 26 17:50:17.472059 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:50:17.471904    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 26 17:50:17.472059 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:50:17.471957    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/28761b10-b1ee-441e-946c-bdd7ad7e9eed-kube-rbac-proxy-cm podName:28761b10-b1ee-441e-946c-bdd7ad7e9eed nodeName:}" failed. No retries permitted until 2026-07-26 17:50:19.471940535 +0000 UTC m=+76.124131384 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/28761b10-b1ee-441e-946c-bdd7ad7e9eed-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kdznm" (UID: "28761b10-b1ee-441e-946c-bdd7ad7e9eed") : configmap "kube-rbac-proxy" not found
Jul 26 17:50:17.715240 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:17.715177    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-974ql_1fcdab4c-350c-44c3-9bc0-a9995d33c539/dns-node-resolver/0.log"
Jul 26 17:50:18.315000 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:18.314963    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-f9b95bd64-5jvnp_0f532906-dddd-459d-b8da-3074eab679b1/registry/0.log"
Jul 26 17:50:18.914975 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:18.914945    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ddfsj_0557c247-c431-4eef-9de0-0b272b9e5250/node-ca/0.log"
Jul 26 17:50:19.116230 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:19.116197    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65db774ff-4whvz_759b21a0-1d29-4328-ad81-1e6c9f0c5b66/router/0.log"
Jul 26 17:50:19.489450 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:19.489427    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/28761b10-b1ee-441e-946c-bdd7ad7e9eed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:19.489684 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:50:19.489569    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 26 17:50:19.489684 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:19.489582    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/28761b10-b1ee-441e-946c-bdd7ad7e9eed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:19.489684 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:50:19.489642    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28761b10-b1ee-441e-946c-bdd7ad7e9eed-insights-runtime-extractor-tls podName:28761b10-b1ee-441e-946c-bdd7ad7e9eed nodeName:}" failed. No retries permitted until 2026-07-26 17:50:23.48962445 +0000 UTC m=+80.141815296 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/28761b10-b1ee-441e-946c-bdd7ad7e9eed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kdznm" (UID: "28761b10-b1ee-441e-946c-bdd7ad7e9eed") : secret "insights-runtime-extractor-tls" not found
Jul 26 17:50:19.489785 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:50:19.489686    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 26 17:50:19.489785 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:50:19.489739    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/28761b10-b1ee-441e-946c-bdd7ad7e9eed-kube-rbac-proxy-cm podName:28761b10-b1ee-441e-946c-bdd7ad7e9eed nodeName:}" failed. No retries permitted until 2026-07-26 17:50:23.489725988 +0000 UTC m=+80.141916825 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/28761b10-b1ee-441e-946c-bdd7ad7e9eed-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kdznm" (UID: "28761b10-b1ee-441e-946c-bdd7ad7e9eed") : configmap "kube-rbac-proxy" not found
Jul 26 17:50:19.515000 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:19.514974    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c9hsm_3321b850-63be-4345-ac65-afaee093369a/serve-healthcheck-canary/0.log"
Jul 26 17:50:20.275648 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:20.275612    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-558cb5c8df-jwwxj" event={"ID":"b6e8576d-3999-4c2b-b503-ee302a2650a7","Type":"ContainerStarted","Data":"2bf6c0c361fbde86b581b7e98aa0d70397f2b62beffd7fba5fad3a3478de9488"}
Jul 26 17:50:20.275865 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:20.275778    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-558cb5c8df-jwwxj"
Jul 26 17:50:20.277652 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:20.277630    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-558cb5c8df-jwwxj"
Jul 26 17:50:20.295457 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:20.295397    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-558cb5c8df-jwwxj" podStartSLOduration=1.701755881 podStartE2EDuration="11.29538456s" totalImagesPullingTime="9.593628679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:50:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:20.294489616 +0000 UTC m=+76.946680473" watchObservedRunningTime="2026-07-26 17:50:20.29538456 +0000 UTC m=+76.947575447"
Jul 26 17:50:22.832302 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:22.832269    2570 scope.go:117] "RemoveContainer" containerID="a2b1d48397d8620a3f9fe18c5e1931212d62465328dc1f05e572ffd9ecb335b6"
Jul 26 17:50:22.832654 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:50:22.832424    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-q8vbm_openshift-console-operator(801b12f8-cc05-4646-99a0-304297466b26)\"" pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm" podUID="801b12f8-cc05-4646-99a0-304297466b26"
Jul 26 17:50:23.521754 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:23.521695    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/28761b10-b1ee-441e-946c-bdd7ad7e9eed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:23.521942 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:23.521809    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/28761b10-b1ee-441e-946c-bdd7ad7e9eed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:23.521942 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:50:23.521818    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 26 17:50:23.521942 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:50:23.521884    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/28761b10-b1ee-441e-946c-bdd7ad7e9eed-kube-rbac-proxy-cm podName:28761b10-b1ee-441e-946c-bdd7ad7e9eed nodeName:}" failed. No retries permitted until 2026-07-26 17:50:31.521867627 +0000 UTC m=+88.174058476 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/28761b10-b1ee-441e-946c-bdd7ad7e9eed-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kdznm" (UID: "28761b10-b1ee-441e-946c-bdd7ad7e9eed") : configmap "kube-rbac-proxy" not found
Jul 26 17:50:23.522152 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:50:23.521943    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 26 17:50:23.522152 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:50:23.522016    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28761b10-b1ee-441e-946c-bdd7ad7e9eed-insights-runtime-extractor-tls podName:28761b10-b1ee-441e-946c-bdd7ad7e9eed nodeName:}" failed. No retries permitted until 2026-07-26 17:50:31.52200163 +0000 UTC m=+88.174192465 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/28761b10-b1ee-441e-946c-bdd7ad7e9eed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kdznm" (UID: "28761b10-b1ee-441e-946c-bdd7ad7e9eed") : secret "insights-runtime-extractor-tls" not found
Jul 26 17:50:25.529904 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.529869    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-znqkg"]
Jul 26 17:50:25.533334 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.533317    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.536523 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.536499    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fs6k7\""
Jul 26 17:50:25.536729 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.536701    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 26 17:50:25.536829 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.536736    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 26 17:50:25.536829 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.536799    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 26 17:50:25.537050 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.537033    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 26 17:50:25.537137 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.537076    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 26 17:50:25.537194 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.537179    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 26 17:50:25.640019 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.639986    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e120f5e8-a16c-4e04-88c6-993fafdc7a40-node-exporter-accelerators-collector-config\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.640019 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.640019    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e120f5e8-a16c-4e04-88c6-993fafdc7a40-node-exporter-tls\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.640258 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.640043    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e120f5e8-a16c-4e04-88c6-993fafdc7a40-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.640258 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.640074    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e120f5e8-a16c-4e04-88c6-993fafdc7a40-root\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.640258 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.640164    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e120f5e8-a16c-4e04-88c6-993fafdc7a40-sys\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.640258 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.640199    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e120f5e8-a16c-4e04-88c6-993fafdc7a40-node-exporter-textfile\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.640258 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.640227    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e120f5e8-a16c-4e04-88c6-993fafdc7a40-metrics-client-ca\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.640258 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.640242    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6jzb\" (UniqueName: \"kubernetes.io/projected/e120f5e8-a16c-4e04-88c6-993fafdc7a40-kube-api-access-f6jzb\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.640435 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.640266    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e120f5e8-a16c-4e04-88c6-993fafdc7a40-node-exporter-wtmp\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.741305 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.741277    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e120f5e8-a16c-4e04-88c6-993fafdc7a40-root\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.741469 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.741323    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e120f5e8-a16c-4e04-88c6-993fafdc7a40-sys\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.741469 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.741379    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e120f5e8-a16c-4e04-88c6-993fafdc7a40-sys\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.741469 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.741400    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e120f5e8-a16c-4e04-88c6-993fafdc7a40-root\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.741469 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.741442    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e120f5e8-a16c-4e04-88c6-993fafdc7a40-node-exporter-textfile\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.741626 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.741480    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e120f5e8-a16c-4e04-88c6-993fafdc7a40-metrics-client-ca\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.741626 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.741497    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6jzb\" (UniqueName: \"kubernetes.io/projected/e120f5e8-a16c-4e04-88c6-993fafdc7a40-kube-api-access-f6jzb\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.741626 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.741519    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e120f5e8-a16c-4e04-88c6-993fafdc7a40-node-exporter-wtmp\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.741626 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.741561    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e120f5e8-a16c-4e04-88c6-993fafdc7a40-node-exporter-accelerators-collector-config\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.741626 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.741586    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e120f5e8-a16c-4e04-88c6-993fafdc7a40-node-exporter-tls\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.741626 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.741609    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e120f5e8-a16c-4e04-88c6-993fafdc7a40-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.741904 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.741845    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e120f5e8-a16c-4e04-88c6-993fafdc7a40-node-exporter-textfile\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.741963 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.741934    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e120f5e8-a16c-4e04-88c6-993fafdc7a40-node-exporter-wtmp\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.742212 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.742192    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e120f5e8-a16c-4e04-88c6-993fafdc7a40-metrics-client-ca\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.742212 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.742200    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e120f5e8-a16c-4e04-88c6-993fafdc7a40-node-exporter-accelerators-collector-config\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.743970 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.743953    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e120f5e8-a16c-4e04-88c6-993fafdc7a40-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.744130 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.744101    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e120f5e8-a16c-4e04-88c6-993fafdc7a40-node-exporter-tls\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.748250 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.748231    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6jzb\" (UniqueName: \"kubernetes.io/projected/e120f5e8-a16c-4e04-88c6-993fafdc7a40-kube-api-access-f6jzb\") pod \"node-exporter-znqkg\" (UID: \"e120f5e8-a16c-4e04-88c6-993fafdc7a40\") " pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.842380 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:25.842296    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-znqkg"
Jul 26 17:50:25.854501 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:50:25.854476    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode120f5e8_a16c_4e04_88c6_993fafdc7a40.slice/crio-819aa1754fe59f34c95bc589c173f26eaa075ebeb4f62a1c17842f2287cf0f28 WatchSource:0}: Error finding container 819aa1754fe59f34c95bc589c173f26eaa075ebeb4f62a1c17842f2287cf0f28: Status 404 returned error can't find the container with id 819aa1754fe59f34c95bc589c173f26eaa075ebeb4f62a1c17842f2287cf0f28
Jul 26 17:50:26.299328 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:26.299288    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-znqkg" event={"ID":"e120f5e8-a16c-4e04-88c6-993fafdc7a40","Type":"ContainerStarted","Data":"819aa1754fe59f34c95bc589c173f26eaa075ebeb4f62a1c17842f2287cf0f28"}
Jul 26 17:50:27.269516 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:27.269486    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-26rxm"
Jul 26 17:50:27.305602 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:27.305569    2570 generic.go:356] "Generic (PLEG): container finished" podID="e120f5e8-a16c-4e04-88c6-993fafdc7a40" containerID="903b4cac7c1be8af82c3e18ec7428e223fba4de6b58ef2fa593a85682484cd78" exitCode=0
Jul 26 17:50:27.305856 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:27.305652    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-znqkg" event={"ID":"e120f5e8-a16c-4e04-88c6-993fafdc7a40","Type":"ContainerDied","Data":"903b4cac7c1be8af82c3e18ec7428e223fba4de6b58ef2fa593a85682484cd78"}
Jul 26 17:50:28.311572 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:28.311536    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-znqkg" event={"ID":"e120f5e8-a16c-4e04-88c6-993fafdc7a40","Type":"ContainerStarted","Data":"c2c6c80ae1b9f4cdc78f53e0fd89bcaaf272b7e0e21df916e9521e2fdba55226"}
Jul 26 17:50:28.311957 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:28.311587    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-znqkg" event={"ID":"e120f5e8-a16c-4e04-88c6-993fafdc7a40","Type":"ContainerStarted","Data":"871e5015383261ccb9e26239931b75000f6d75903c289353106cb6d1a8ad83bf"}
Jul 26 17:50:28.330129 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:28.330052    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-znqkg" podStartSLOduration=2.379408588 podStartE2EDuration="3.330034722s" totalImagesPullingTime="885.423568ms" totalInitContainerRuntime="65.202566ms" isStatefulPod=true podCreationTimestamp="2026-07-26 17:50:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:28.32915136 +0000 UTC m=+84.981342224" watchObservedRunningTime="2026-07-26 17:50:28.330034722 +0000 UTC m=+84.982225580"
Jul 26 17:50:31.330984 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:31.330945    2570 patch_prober.go:28] interesting pod/image-registry-f9b95bd64-5jvnp container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 26 17:50:31.331378 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:31.330999    2570 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp" podUID="0f532906-dddd-459d-b8da-3074eab679b1" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 26 17:50:31.595399 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:31.595287    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/28761b10-b1ee-441e-946c-bdd7ad7e9eed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:31.595579 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:31.595413    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/28761b10-b1ee-441e-946c-bdd7ad7e9eed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:31.595892 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:31.595871    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/28761b10-b1ee-441e-946c-bdd7ad7e9eed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:31.597652 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:31.597630    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/28761b10-b1ee-441e-946c-bdd7ad7e9eed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kdznm\" (UID: \"28761b10-b1ee-441e-946c-bdd7ad7e9eed\") " pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:31.653809 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:31.653777    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kdznm"
Jul 26 17:50:31.785808 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:31.785771    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kdznm"]
Jul 26 17:50:31.795967 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:31.795936    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-f9b95bd64-5jvnp"]
Jul 26 17:50:31.800139 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:31.800099    2570 patch_prober.go:28] interesting pod/image-registry-f9b95bd64-5jvnp container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 26 17:50:31.800251 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:31.800165    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp" podUID="0f532906-dddd-459d-b8da-3074eab679b1" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 26 17:50:32.328248 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:32.328154    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kdznm" event={"ID":"28761b10-b1ee-441e-946c-bdd7ad7e9eed","Type":"ContainerStarted","Data":"45b5967117c548d693a969ae4798f19f7546e5ca934bc7b05b5c7f9738b677ea"}
Jul 26 17:50:35.832953 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:35.832920    2570 scope.go:117] "RemoveContainer" containerID="a2b1d48397d8620a3f9fe18c5e1931212d62465328dc1f05e572ffd9ecb335b6"
Jul 26 17:50:37.198759 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:37.198727    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-6mtw5"]
Jul 26 17:50:37.202200 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:37.202175    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-6mtw5"
Jul 26 17:50:37.204492 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:37.204469    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-zd8f5\""
Jul 26 17:50:37.204599 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:37.204511    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 26 17:50:37.204599 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:37.204469    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 26 17:50:37.214756 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:37.214732    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-6mtw5"]
Jul 26 17:50:37.246743 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:37.246712    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmwtl\" (UniqueName: \"kubernetes.io/projected/2a2f2007-9f70-43a5-9481-decd359f701d-kube-api-access-pmwtl\") pod \"downloads-7c97dc4555-6mtw5\" (UID: \"2a2f2007-9f70-43a5-9481-decd359f701d\") " pod="openshift-console/downloads-7c97dc4555-6mtw5"
Jul 26 17:50:37.348183 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:37.347984    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmwtl\" (UniqueName: \"kubernetes.io/projected/2a2f2007-9f70-43a5-9481-decd359f701d-kube-api-access-pmwtl\") pod \"downloads-7c97dc4555-6mtw5\" (UID: \"2a2f2007-9f70-43a5-9481-decd359f701d\") " pod="openshift-console/downloads-7c97dc4555-6mtw5"
Jul 26 17:50:37.351964 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:37.351938    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 17:50:37.352530 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:37.352073    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm" event={"ID":"801b12f8-cc05-4646-99a0-304297466b26","Type":"ContainerStarted","Data":"3ad15302059c28235f3f69c08224899c295d0cd074ce5d02bcbaf5172fd26a7b"}
Jul 26 17:50:37.352530 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:37.352397    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm"
Jul 26 17:50:37.355831 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:37.355161    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kdznm" event={"ID":"28761b10-b1ee-441e-946c-bdd7ad7e9eed","Type":"ContainerStarted","Data":"decad95122f5f9116ebfdb7dfeb2ebf728cd59196b35bed9708b9111fe7a7d0d"}
Jul 26 17:50:37.357099 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:37.357038    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmwtl\" (UniqueName: \"kubernetes.io/projected/2a2f2007-9f70-43a5-9481-decd359f701d-kube-api-access-pmwtl\") pod \"downloads-7c97dc4555-6mtw5\" (UID: \"2a2f2007-9f70-43a5-9481-decd359f701d\") " pod="openshift-console/downloads-7c97dc4555-6mtw5"
Jul 26 17:50:37.358700 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:37.358678    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm"
Jul 26 17:50:37.372792 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:37.372742    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-q8vbm" podStartSLOduration=83.592644117 podStartE2EDuration="1m30.372728085s" totalImagesPullingTime="6.780083968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:49:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:37.372123795 +0000 UTC m=+94.024314644" watchObservedRunningTime="2026-07-26 17:50:37.372728085 +0000 UTC m=+94.024918942"
Jul 26 17:50:37.516661 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:37.516631    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-6mtw5"
Jul 26 17:50:37.665906 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:37.665871    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-6mtw5"]
Jul 26 17:50:37.669167 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:50:37.669134    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a2f2007_9f70_43a5_9481_decd359f701d.slice/crio-e49a1fe7248cac2da53fac152ae9edb2f09d1662db36cb6320e2dd8a5efdceab WatchSource:0}: Error finding container e49a1fe7248cac2da53fac152ae9edb2f09d1662db36cb6320e2dd8a5efdceab: Status 404 returned error can't find the container with id e49a1fe7248cac2da53fac152ae9edb2f09d1662db36cb6320e2dd8a5efdceab
Jul 26 17:50:38.361602 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:38.361528    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-6mtw5" event={"ID":"2a2f2007-9f70-43a5-9481-decd359f701d","Type":"ContainerStarted","Data":"e49a1fe7248cac2da53fac152ae9edb2f09d1662db36cb6320e2dd8a5efdceab"}
Jul 26 17:50:41.224554 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:41.224518    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-h8sjb"
Jul 26 17:50:41.377593 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:41.377548    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kdznm" event={"ID":"28761b10-b1ee-441e-946c-bdd7ad7e9eed","Type":"ContainerStarted","Data":"3e87b155d0d0b7972d9f09d9dd73e3e11f772f0623ecaf1768135b26379ce534"}
Jul 26 17:50:41.801911 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:41.801879    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:50:41.915703 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:41.915669    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-785dbf9c78-kqs5s"]
Jul 26 17:50:41.919275 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:41.919247    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-785dbf9c78-kqs5s"
Jul 26 17:50:41.921589 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:41.921562    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 26 17:50:41.921720 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:41.921598    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 26 17:50:41.921720 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:41.921695    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-xzhcr\""
Jul 26 17:50:41.921819 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:41.921804    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 26 17:50:41.922205 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:41.921945    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 26 17:50:41.922205 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:41.922012    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 26 17:50:41.926956 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:41.926932    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-785dbf9c78-kqs5s"]
Jul 26 17:50:41.987642 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:41.987597    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e304865e-977a-4ca7-8518-4ddfd0849359-oauth-serving-cert\") pod \"console-785dbf9c78-kqs5s\" (UID: \"e304865e-977a-4ca7-8518-4ddfd0849359\") " pod="openshift-console/console-785dbf9c78-kqs5s"
Jul 26 17:50:41.987642 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:41.987636    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvrnq\" (UniqueName: \"kubernetes.io/projected/e304865e-977a-4ca7-8518-4ddfd0849359-kube-api-access-bvrnq\") pod \"console-785dbf9c78-kqs5s\" (UID: \"e304865e-977a-4ca7-8518-4ddfd0849359\") " pod="openshift-console/console-785dbf9c78-kqs5s"
Jul 26 17:50:41.987898 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:41.987749    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e304865e-977a-4ca7-8518-4ddfd0849359-console-oauth-config\") pod \"console-785dbf9c78-kqs5s\" (UID: \"e304865e-977a-4ca7-8518-4ddfd0849359\") " pod="openshift-console/console-785dbf9c78-kqs5s"
Jul 26 17:50:41.987898 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:41.987780    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e304865e-977a-4ca7-8518-4ddfd0849359-console-serving-cert\") pod \"console-785dbf9c78-kqs5s\" (UID: \"e304865e-977a-4ca7-8518-4ddfd0849359\") " pod="openshift-console/console-785dbf9c78-kqs5s"
Jul 26 17:50:41.987898 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:41.987801    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e304865e-977a-4ca7-8518-4ddfd0849359-console-config\") pod \"console-785dbf9c78-kqs5s\" (UID: \"e304865e-977a-4ca7-8518-4ddfd0849359\") " pod="openshift-console/console-785dbf9c78-kqs5s"
Jul 26 17:50:41.987898 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:41.987832    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e304865e-977a-4ca7-8518-4ddfd0849359-service-ca\") pod \"console-785dbf9c78-kqs5s\" (UID: \"e304865e-977a-4ca7-8518-4ddfd0849359\") " pod="openshift-console/console-785dbf9c78-kqs5s"
Jul 26 17:50:42.088744 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:42.088661    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e304865e-977a-4ca7-8518-4ddfd0849359-console-oauth-config\") pod \"console-785dbf9c78-kqs5s\" (UID: \"e304865e-977a-4ca7-8518-4ddfd0849359\") " pod="openshift-console/console-785dbf9c78-kqs5s"
Jul 26 17:50:42.088744 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:42.088699    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e304865e-977a-4ca7-8518-4ddfd0849359-console-serving-cert\") pod \"console-785dbf9c78-kqs5s\" (UID: \"e304865e-977a-4ca7-8518-4ddfd0849359\") " pod="openshift-console/console-785dbf9c78-kqs5s"
Jul 26 17:50:42.088744 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:42.088717    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e304865e-977a-4ca7-8518-4ddfd0849359-console-config\") pod \"console-785dbf9c78-kqs5s\" (UID: \"e304865e-977a-4ca7-8518-4ddfd0849359\") " pod="openshift-console/console-785dbf9c78-kqs5s"
Jul 26 17:50:42.088999 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:42.088935    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e304865e-977a-4ca7-8518-4ddfd0849359-service-ca\") pod \"console-785dbf9c78-kqs5s\" (UID: \"e304865e-977a-4ca7-8518-4ddfd0849359\") " pod="openshift-console/console-785dbf9c78-kqs5s"
Jul 26 17:50:42.088999 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:42.088992    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e304865e-977a-4ca7-8518-4ddfd0849359-oauth-serving-cert\") pod \"console-785dbf9c78-kqs5s\" (UID: \"e304865e-977a-4ca7-8518-4ddfd0849359\") " pod="openshift-console/console-785dbf9c78-kqs5s"
Jul 26 17:50:42.089100 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:42.089038    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvrnq\" (UniqueName: \"kubernetes.io/projected/e304865e-977a-4ca7-8518-4ddfd0849359-kube-api-access-bvrnq\") pod \"console-785dbf9c78-kqs5s\" (UID: \"e304865e-977a-4ca7-8518-4ddfd0849359\") " pod="openshift-console/console-785dbf9c78-kqs5s"
Jul 26 17:50:42.089586 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:42.089557    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e304865e-977a-4ca7-8518-4ddfd0849359-console-config\") pod \"console-785dbf9c78-kqs5s\" (UID: \"e304865e-977a-4ca7-8518-4ddfd0849359\") " pod="openshift-console/console-785dbf9c78-kqs5s"
Jul 26 17:50:42.089705 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:42.089663    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e304865e-977a-4ca7-8518-4ddfd0849359-service-ca\") pod \"console-785dbf9c78-kqs5s\" (UID: \"e304865e-977a-4ca7-8518-4ddfd0849359\") " pod="openshift-console/console-785dbf9c78-kqs5s"
Jul 26 17:50:42.089705 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:42.089663    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e304865e-977a-4ca7-8518-4ddfd0849359-oauth-serving-cert\") pod \"console-785dbf9c78-kqs5s\" (UID: \"e304865e-977a-4ca7-8518-4ddfd0849359\") " pod="openshift-console/console-785dbf9c78-kqs5s"
Jul 26 17:50:42.091617 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:42.091589    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e304865e-977a-4ca7-8518-4ddfd0849359-console-serving-cert\") pod \"console-785dbf9c78-kqs5s\" (UID: \"e304865e-977a-4ca7-8518-4ddfd0849359\") " pod="openshift-console/console-785dbf9c78-kqs5s"
Jul 26 17:50:42.091897 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:42.091880    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e304865e-977a-4ca7-8518-4ddfd0849359-console-oauth-config\") pod \"console-785dbf9c78-kqs5s\" (UID: \"e304865e-977a-4ca7-8518-4ddfd0849359\") " pod="openshift-console/console-785dbf9c78-kqs5s"
Jul 26 17:50:42.096806 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:42.096783    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvrnq\" (UniqueName: \"kubernetes.io/projected/e304865e-977a-4ca7-8518-4ddfd0849359-kube-api-access-bvrnq\") pod \"console-785dbf9c78-kqs5s\" (UID: \"e304865e-977a-4ca7-8518-4ddfd0849359\") " pod="openshift-console/console-785dbf9c78-kqs5s"
Jul 26 17:50:42.232924 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:42.232864    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-785dbf9c78-kqs5s"
Jul 26 17:50:42.402323 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:42.402246    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-785dbf9c78-kqs5s"]
Jul 26 17:50:42.404940 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:50:42.404902    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode304865e_977a_4ca7_8518_4ddfd0849359.slice/crio-07913e3aed65936529d296ea17e0c020904dc8e853c0bdce622994244c872ed1 WatchSource:0}: Error finding container 07913e3aed65936529d296ea17e0c020904dc8e853c0bdce622994244c872ed1: Status 404 returned error can't find the container with id 07913e3aed65936529d296ea17e0c020904dc8e853c0bdce622994244c872ed1
Jul 26 17:50:43.389982 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:43.389930    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-785dbf9c78-kqs5s" event={"ID":"e304865e-977a-4ca7-8518-4ddfd0849359","Type":"ContainerStarted","Data":"07913e3aed65936529d296ea17e0c020904dc8e853c0bdce622994244c872ed1"}
Jul 26 17:50:46.406789 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:46.406692    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kdznm" event={"ID":"28761b10-b1ee-441e-946c-bdd7ad7e9eed","Type":"ContainerStarted","Data":"07b38eff5cbc2756bbe9db98a662988557a42931d68d7aefaf891e274290309f"}
Jul 26 17:50:46.428543 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:46.428486    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kdznm" podStartSLOduration=18.059302005 podStartE2EDuration="31.42846693s" totalImagesPullingTime="13.369164925s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:50:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:46.427925257 +0000 UTC m=+103.080116116" watchObservedRunningTime="2026-07-26 17:50:46.42846693 +0000 UTC m=+103.080657789"
Jul 26 17:50:49.298726 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.298692    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b785b5d76-zcrkd"]
Jul 26 17:50:49.330410 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.330375    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b785b5d76-zcrkd"]
Jul 26 17:50:49.330563 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.330503    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:49.339361 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.339327    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 26 17:50:49.455912 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.455872    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4qv8\" (UniqueName: \"kubernetes.io/projected/472f1421-a4fb-4a72-a2f3-46b463b71e53-kube-api-access-p4qv8\") pod \"console-5b785b5d76-zcrkd\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") " pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:49.456103 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.455924    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/472f1421-a4fb-4a72-a2f3-46b463b71e53-trusted-ca-bundle\") pod \"console-5b785b5d76-zcrkd\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") " pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:49.456103 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.456012    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/472f1421-a4fb-4a72-a2f3-46b463b71e53-service-ca\") pod \"console-5b785b5d76-zcrkd\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") " pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:49.456103 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.456046    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/472f1421-a4fb-4a72-a2f3-46b463b71e53-oauth-serving-cert\") pod \"console-5b785b5d76-zcrkd\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") " pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:49.456103 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.456074    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/472f1421-a4fb-4a72-a2f3-46b463b71e53-console-config\") pod \"console-5b785b5d76-zcrkd\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") " pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:49.456307 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.456106    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/472f1421-a4fb-4a72-a2f3-46b463b71e53-console-oauth-config\") pod \"console-5b785b5d76-zcrkd\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") " pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:49.456307 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.456160    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/472f1421-a4fb-4a72-a2f3-46b463b71e53-console-serving-cert\") pod \"console-5b785b5d76-zcrkd\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") " pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:49.556823 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.556742    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/472f1421-a4fb-4a72-a2f3-46b463b71e53-oauth-serving-cert\") pod \"console-5b785b5d76-zcrkd\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") " pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:49.556823 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.556790    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/472f1421-a4fb-4a72-a2f3-46b463b71e53-console-config\") pod \"console-5b785b5d76-zcrkd\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") " pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:49.556823 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.556815    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/472f1421-a4fb-4a72-a2f3-46b463b71e53-console-oauth-config\") pod \"console-5b785b5d76-zcrkd\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") " pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:49.557102 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.556847    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/472f1421-a4fb-4a72-a2f3-46b463b71e53-console-serving-cert\") pod \"console-5b785b5d76-zcrkd\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") " pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:49.557102 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.556905    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4qv8\" (UniqueName: \"kubernetes.io/projected/472f1421-a4fb-4a72-a2f3-46b463b71e53-kube-api-access-p4qv8\") pod \"console-5b785b5d76-zcrkd\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") " pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:49.557102 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.556969    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/472f1421-a4fb-4a72-a2f3-46b463b71e53-trusted-ca-bundle\") pod \"console-5b785b5d76-zcrkd\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") " pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:49.557102 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.557024    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/472f1421-a4fb-4a72-a2f3-46b463b71e53-service-ca\") pod \"console-5b785b5d76-zcrkd\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") " pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:49.557605 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.557556    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/472f1421-a4fb-4a72-a2f3-46b463b71e53-console-config\") pod \"console-5b785b5d76-zcrkd\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") " pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:49.557970 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.557945    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/472f1421-a4fb-4a72-a2f3-46b463b71e53-service-ca\") pod \"console-5b785b5d76-zcrkd\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") " pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:49.558087 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.557992    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/472f1421-a4fb-4a72-a2f3-46b463b71e53-oauth-serving-cert\") pod \"console-5b785b5d76-zcrkd\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") " pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:49.558699 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.558676    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/472f1421-a4fb-4a72-a2f3-46b463b71e53-trusted-ca-bundle\") pod \"console-5b785b5d76-zcrkd\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") " pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:49.559636 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.559616    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/472f1421-a4fb-4a72-a2f3-46b463b71e53-console-oauth-config\") pod \"console-5b785b5d76-zcrkd\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") " pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:49.559723 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.559710    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/472f1421-a4fb-4a72-a2f3-46b463b71e53-console-serving-cert\") pod \"console-5b785b5d76-zcrkd\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") " pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:49.564868 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.564848    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4qv8\" (UniqueName: \"kubernetes.io/projected/472f1421-a4fb-4a72-a2f3-46b463b71e53-kube-api-access-p4qv8\") pod \"console-5b785b5d76-zcrkd\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") " pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:49.642907 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:49.642870    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:55.468704 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:55.468502    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b785b5d76-zcrkd"]
Jul 26 17:50:55.471668 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:50:55.471639    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod472f1421_a4fb_4a72_a2f3_46b463b71e53.slice/crio-06e4fed72e559144754ca5d735079b91628e78f76ea3410df187eb99d6e9148f WatchSource:0}: Error finding container 06e4fed72e559144754ca5d735079b91628e78f76ea3410df187eb99d6e9148f: Status 404 returned error can't find the container with id 06e4fed72e559144754ca5d735079b91628e78f76ea3410df187eb99d6e9148f
Jul 26 17:50:56.452418 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:56.452378    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-6mtw5" event={"ID":"2a2f2007-9f70-43a5-9481-decd359f701d","Type":"ContainerStarted","Data":"acff6d38873804723df43f49235308aef5c887b17cc7f9aeb5e14e039aab789c"}
Jul 26 17:50:56.452624 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:56.452557    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-6mtw5"
Jul 26 17:50:56.454814 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:56.454787    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b785b5d76-zcrkd" event={"ID":"472f1421-a4fb-4a72-a2f3-46b463b71e53","Type":"ContainerStarted","Data":"1b68e3c1fc0a7998f7288e9655c626af9e9bb90b1098c38ae2c1795e83638122"}
Jul 26 17:50:56.454937 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:56.454837    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b785b5d76-zcrkd" event={"ID":"472f1421-a4fb-4a72-a2f3-46b463b71e53","Type":"ContainerStarted","Data":"06e4fed72e559144754ca5d735079b91628e78f76ea3410df187eb99d6e9148f"}
Jul 26 17:50:56.456480 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:56.456450    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-785dbf9c78-kqs5s" event={"ID":"e304865e-977a-4ca7-8518-4ddfd0849359","Type":"ContainerStarted","Data":"783a0eb94f9ba7861af0027a23d574c9c061ece24eee9ead27667df0c0f6e607"}
Jul 26 17:50:56.470274 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:56.470249    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-6mtw5"
Jul 26 17:50:56.483017 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:56.482972    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-6mtw5" podStartSLOduration=1.481079533 podStartE2EDuration="19.482954493s" totalImagesPullingTime="18.00187496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:50:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:56.482499791 +0000 UTC m=+113.134690650" watchObservedRunningTime="2026-07-26 17:50:56.482954493 +0000 UTC m=+113.135145353"
Jul 26 17:50:56.507254 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:56.507194    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-785dbf9c78-kqs5s" podStartSLOduration=2.57693236 podStartE2EDuration="15.507172614s" totalImagesPullingTime="12.930240254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:50:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:56.506463369 +0000 UTC m=+113.158654228" watchObservedRunningTime="2026-07-26 17:50:56.507172614 +0000 UTC m=+113.159363474"
Jul 26 17:50:56.526272 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:56.526210    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b785b5d76-zcrkd" podStartSLOduration=7.526191956 podStartE2EDuration="7.526191956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:50:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:56.525927221 +0000 UTC m=+113.178118080" watchObservedRunningTime="2026-07-26 17:50:56.526191956 +0000 UTC m=+113.178382815"
Jul 26 17:50:56.815309 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:56.815259    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp" podUID="0f532906-dddd-459d-b8da-3074eab679b1" containerName="registry" containerID="cri-o://a65e029c7dc115ee36358b94bd24d0fda180f19cd43a5bf8456ec7cd721ff6cc" gracePeriod=30
Jul 26 17:50:57.127739 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.127707    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:50:57.231410 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.231365    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-bound-sa-token\") pod \"0f532906-dddd-459d-b8da-3074eab679b1\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") "
Jul 26 17:50:57.231606 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.231458    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f532906-dddd-459d-b8da-3074eab679b1-trusted-ca\") pod \"0f532906-dddd-459d-b8da-3074eab679b1\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") "
Jul 26 17:50:57.231606 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.231500    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h554k\" (UniqueName: \"kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-kube-api-access-h554k\") pod \"0f532906-dddd-459d-b8da-3074eab679b1\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") "
Jul 26 17:50:57.231606 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.231551    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f532906-dddd-459d-b8da-3074eab679b1-installation-pull-secrets\") pod \"0f532906-dddd-459d-b8da-3074eab679b1\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") "
Jul 26 17:50:57.231606 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.231582    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f532906-dddd-459d-b8da-3074eab679b1-image-registry-private-configuration\") pod \"0f532906-dddd-459d-b8da-3074eab679b1\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") "
Jul 26 17:50:57.231834 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.231625    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-registry-tls\") pod \"0f532906-dddd-459d-b8da-3074eab679b1\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") "
Jul 26 17:50:57.231834 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.231653    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f532906-dddd-459d-b8da-3074eab679b1-ca-trust-extracted\") pod \"0f532906-dddd-459d-b8da-3074eab679b1\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") "
Jul 26 17:50:57.231834 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.231693    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f532906-dddd-459d-b8da-3074eab679b1-registry-certificates\") pod \"0f532906-dddd-459d-b8da-3074eab679b1\" (UID: \"0f532906-dddd-459d-b8da-3074eab679b1\") "
Jul 26 17:50:57.231982 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.231891    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f532906-dddd-459d-b8da-3074eab679b1-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0f532906-dddd-459d-b8da-3074eab679b1" (UID: "0f532906-dddd-459d-b8da-3074eab679b1"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:50:57.232379 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.232338    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f532906-dddd-459d-b8da-3074eab679b1-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0f532906-dddd-459d-b8da-3074eab679b1" (UID: "0f532906-dddd-459d-b8da-3074eab679b1"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:50:57.235089 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.235039    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f532906-dddd-459d-b8da-3074eab679b1-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0f532906-dddd-459d-b8da-3074eab679b1" (UID: "0f532906-dddd-459d-b8da-3074eab679b1"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 17:50:57.235229 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.235192    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0f532906-dddd-459d-b8da-3074eab679b1" (UID: "0f532906-dddd-459d-b8da-3074eab679b1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:50:57.235584 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.235555    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f532906-dddd-459d-b8da-3074eab679b1-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0f532906-dddd-459d-b8da-3074eab679b1" (UID: "0f532906-dddd-459d-b8da-3074eab679b1"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 17:50:57.236667 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.236641    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0f532906-dddd-459d-b8da-3074eab679b1" (UID: "0f532906-dddd-459d-b8da-3074eab679b1"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:50:57.238149 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.238107    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-kube-api-access-h554k" (OuterVolumeSpecName: "kube-api-access-h554k") pod "0f532906-dddd-459d-b8da-3074eab679b1" (UID: "0f532906-dddd-459d-b8da-3074eab679b1"). InnerVolumeSpecName "kube-api-access-h554k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:50:57.242947 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.242918    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f532906-dddd-459d-b8da-3074eab679b1-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0f532906-dddd-459d-b8da-3074eab679b1" (UID: "0f532906-dddd-459d-b8da-3074eab679b1"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 17:50:57.332903 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.332861    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h554k\" (UniqueName: \"kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-kube-api-access-h554k\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:50:57.332903 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.332900    2570 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f532906-dddd-459d-b8da-3074eab679b1-installation-pull-secrets\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:50:57.333142 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.332917    2570 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f532906-dddd-459d-b8da-3074eab679b1-image-registry-private-configuration\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:50:57.333142 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.332931    2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-registry-tls\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:50:57.333142 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.332944    2570 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f532906-dddd-459d-b8da-3074eab679b1-ca-trust-extracted\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:50:57.333142 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.332958    2570 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f532906-dddd-459d-b8da-3074eab679b1-registry-certificates\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:50:57.333142 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.332970    2570 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f532906-dddd-459d-b8da-3074eab679b1-bound-sa-token\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:50:57.333142 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.332980    2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f532906-dddd-459d-b8da-3074eab679b1-trusted-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:50:57.467066 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.467028    2570 generic.go:356] "Generic (PLEG): container finished" podID="0f532906-dddd-459d-b8da-3074eab679b1" containerID="a65e029c7dc115ee36358b94bd24d0fda180f19cd43a5bf8456ec7cd721ff6cc" exitCode=0
Jul 26 17:50:57.467263 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.467084    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp"
Jul 26 17:50:57.467263 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.467144    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp" event={"ID":"0f532906-dddd-459d-b8da-3074eab679b1","Type":"ContainerDied","Data":"a65e029c7dc115ee36358b94bd24d0fda180f19cd43a5bf8456ec7cd721ff6cc"}
Jul 26 17:50:57.467263 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.467217    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f9b95bd64-5jvnp" event={"ID":"0f532906-dddd-459d-b8da-3074eab679b1","Type":"ContainerDied","Data":"3a8eeb1f39e760a4b284d34869c8f4fa1630478daf1b089ff02c449617852094"}
Jul 26 17:50:57.467263 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.467253    2570 scope.go:117] "RemoveContainer" containerID="a65e029c7dc115ee36358b94bd24d0fda180f19cd43a5bf8456ec7cd721ff6cc"
Jul 26 17:50:57.477497 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.477474    2570 scope.go:117] "RemoveContainer" containerID="a65e029c7dc115ee36358b94bd24d0fda180f19cd43a5bf8456ec7cd721ff6cc"
Jul 26 17:50:57.477938 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:50:57.477907    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a65e029c7dc115ee36358b94bd24d0fda180f19cd43a5bf8456ec7cd721ff6cc\": container with ID starting with a65e029c7dc115ee36358b94bd24d0fda180f19cd43a5bf8456ec7cd721ff6cc not found: ID does not exist" containerID="a65e029c7dc115ee36358b94bd24d0fda180f19cd43a5bf8456ec7cd721ff6cc"
Jul 26 17:50:57.478057 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.477942    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a65e029c7dc115ee36358b94bd24d0fda180f19cd43a5bf8456ec7cd721ff6cc"} err="failed to get container status \"a65e029c7dc115ee36358b94bd24d0fda180f19cd43a5bf8456ec7cd721ff6cc\": rpc error: code = NotFound desc = could not find container \"a65e029c7dc115ee36358b94bd24d0fda180f19cd43a5bf8456ec7cd721ff6cc\": container with ID starting with a65e029c7dc115ee36358b94bd24d0fda180f19cd43a5bf8456ec7cd721ff6cc not found: ID does not exist"
Jul 26 17:50:57.492678 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.492645    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-f9b95bd64-5jvnp"]
Jul 26 17:50:57.500132 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.498502    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-f9b95bd64-5jvnp"]
Jul 26 17:50:57.839253 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:57.839216    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f532906-dddd-459d-b8da-3074eab679b1" path="/var/lib/kubelet/pods/0f532906-dddd-459d-b8da-3074eab679b1/volumes"
Jul 26 17:50:58.473916 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:58.473881    2570 generic.go:356] "Generic (PLEG): container finished" podID="e6cff45e-fc7b-46bd-a628-a5e5b767269a" containerID="34a67c25c59a7e858bb96cf18cbd91f1fb220d1e159090d3692b9860ee17d3e8" exitCode=0
Jul 26 17:50:58.474130 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:58.473968    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-577j2" event={"ID":"e6cff45e-fc7b-46bd-a628-a5e5b767269a","Type":"ContainerDied","Data":"34a67c25c59a7e858bb96cf18cbd91f1fb220d1e159090d3692b9860ee17d3e8"}
Jul 26 17:50:58.474465 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:58.474444    2570 scope.go:117] "RemoveContainer" containerID="34a67c25c59a7e858bb96cf18cbd91f1fb220d1e159090d3692b9860ee17d3e8"
Jul 26 17:50:59.481726 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:59.481696    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-577j2" event={"ID":"e6cff45e-fc7b-46bd-a628-a5e5b767269a","Type":"ContainerStarted","Data":"5229b06c60739423e1c5dc925ee3b1d3a7a60239638825a20e40ee262285a455"}
Jul 26 17:50:59.643907 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:59.643871    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:59.644090 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:59.643947    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:50:59.649345 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:50:59.649320    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:51:00.490376 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:00.490345    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:51:00.536889 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:00.536847    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-785dbf9c78-kqs5s"]
Jul 26 17:51:02.233424 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:02.233370    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-785dbf9c78-kqs5s"
Jul 26 17:51:07.514907 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:07.514876    2570 generic.go:356] "Generic (PLEG): container finished" podID="40cb8009-e6c4-4fc2-a574-c4ecca296db4" containerID="4038cff4787926ca5b37785c35e778571ad77b8c8242763ac45074bb59bc293a" exitCode=0
Jul 26 17:51:07.515327 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:07.514948    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc" event={"ID":"40cb8009-e6c4-4fc2-a574-c4ecca296db4","Type":"ContainerDied","Data":"4038cff4787926ca5b37785c35e778571ad77b8c8242763ac45074bb59bc293a"}
Jul 26 17:51:07.515327 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:07.515272    2570 scope.go:117] "RemoveContainer" containerID="4038cff4787926ca5b37785c35e778571ad77b8c8242763ac45074bb59bc293a"
Jul 26 17:51:08.520463 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:08.520427    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc" event={"ID":"40cb8009-e6c4-4fc2-a574-c4ecca296db4","Type":"ContainerStarted","Data":"060d1b266397d97d375012122a1d3470c0390fb12f4fa152e9551e2bc42ca2ee"}
Jul 26 17:51:25.563964 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:25.563905    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-785dbf9c78-kqs5s" podUID="e304865e-977a-4ca7-8518-4ddfd0849359" containerName="console" containerID="cri-o://783a0eb94f9ba7861af0027a23d574c9c061ece24eee9ead27667df0c0f6e607" gracePeriod=15
Jul 26 17:51:25.847201 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:25.847169    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-785dbf9c78-kqs5s"
Jul 26 17:51:25.862009 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:25.861987    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e304865e-977a-4ca7-8518-4ddfd0849359-console-serving-cert\") pod \"e304865e-977a-4ca7-8518-4ddfd0849359\" (UID: \"e304865e-977a-4ca7-8518-4ddfd0849359\") "
Jul 26 17:51:25.862157 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:25.862024    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e304865e-977a-4ca7-8518-4ddfd0849359-oauth-serving-cert\") pod \"e304865e-977a-4ca7-8518-4ddfd0849359\" (UID: \"e304865e-977a-4ca7-8518-4ddfd0849359\") "
Jul 26 17:51:25.862157 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:25.862052    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvrnq\" (UniqueName: \"kubernetes.io/projected/e304865e-977a-4ca7-8518-4ddfd0849359-kube-api-access-bvrnq\") pod \"e304865e-977a-4ca7-8518-4ddfd0849359\" (UID: \"e304865e-977a-4ca7-8518-4ddfd0849359\") "
Jul 26 17:51:25.862157 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:25.862093    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e304865e-977a-4ca7-8518-4ddfd0849359-console-config\") pod \"e304865e-977a-4ca7-8518-4ddfd0849359\" (UID: \"e304865e-977a-4ca7-8518-4ddfd0849359\") "
Jul 26 17:51:25.862322 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:25.862165    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e304865e-977a-4ca7-8518-4ddfd0849359-console-oauth-config\") pod \"e304865e-977a-4ca7-8518-4ddfd0849359\" (UID: \"e304865e-977a-4ca7-8518-4ddfd0849359\") "
Jul 26 17:51:25.862322 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:25.862189    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e304865e-977a-4ca7-8518-4ddfd0849359-service-ca\") pod \"e304865e-977a-4ca7-8518-4ddfd0849359\" (UID: \"e304865e-977a-4ca7-8518-4ddfd0849359\") "
Jul 26 17:51:25.862584 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:25.862460    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e304865e-977a-4ca7-8518-4ddfd0849359-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e304865e-977a-4ca7-8518-4ddfd0849359" (UID: "e304865e-977a-4ca7-8518-4ddfd0849359"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:51:25.862584 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:25.862563    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e304865e-977a-4ca7-8518-4ddfd0849359-console-config" (OuterVolumeSpecName: "console-config") pod "e304865e-977a-4ca7-8518-4ddfd0849359" (UID: "e304865e-977a-4ca7-8518-4ddfd0849359"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:51:25.862687 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:25.862622    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e304865e-977a-4ca7-8518-4ddfd0849359-service-ca" (OuterVolumeSpecName: "service-ca") pod "e304865e-977a-4ca7-8518-4ddfd0849359" (UID: "e304865e-977a-4ca7-8518-4ddfd0849359"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:51:25.864400 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:25.864374    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e304865e-977a-4ca7-8518-4ddfd0849359-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e304865e-977a-4ca7-8518-4ddfd0849359" (UID: "e304865e-977a-4ca7-8518-4ddfd0849359"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 17:51:25.864839 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:25.864802    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e304865e-977a-4ca7-8518-4ddfd0849359-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e304865e-977a-4ca7-8518-4ddfd0849359" (UID: "e304865e-977a-4ca7-8518-4ddfd0849359"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 17:51:25.865227 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:25.865200    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e304865e-977a-4ca7-8518-4ddfd0849359-kube-api-access-bvrnq" (OuterVolumeSpecName: "kube-api-access-bvrnq") pod "e304865e-977a-4ca7-8518-4ddfd0849359" (UID: "e304865e-977a-4ca7-8518-4ddfd0849359"). InnerVolumeSpecName "kube-api-access-bvrnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:51:25.963194 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:25.963157    2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e304865e-977a-4ca7-8518-4ddfd0849359-oauth-serving-cert\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:51:25.963194 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:25.963185    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvrnq\" (UniqueName: \"kubernetes.io/projected/e304865e-977a-4ca7-8518-4ddfd0849359-kube-api-access-bvrnq\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:51:25.963194 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:25.963198    2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e304865e-977a-4ca7-8518-4ddfd0849359-console-config\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:51:25.963430 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:25.963209    2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e304865e-977a-4ca7-8518-4ddfd0849359-console-oauth-config\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:51:25.963430 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:25.963221    2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e304865e-977a-4ca7-8518-4ddfd0849359-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:51:25.963430 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:25.963236    2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e304865e-977a-4ca7-8518-4ddfd0849359-console-serving-cert\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:51:26.585798 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:26.585765    2570 generic.go:356] "Generic (PLEG): container finished" podID="e304865e-977a-4ca7-8518-4ddfd0849359" containerID="783a0eb94f9ba7861af0027a23d574c9c061ece24eee9ead27667df0c0f6e607" exitCode=0
Jul 26 17:51:26.586268 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:26.585813    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-785dbf9c78-kqs5s"
Jul 26 17:51:26.586268 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:26.585831    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-785dbf9c78-kqs5s" event={"ID":"e304865e-977a-4ca7-8518-4ddfd0849359","Type":"ContainerDied","Data":"783a0eb94f9ba7861af0027a23d574c9c061ece24eee9ead27667df0c0f6e607"}
Jul 26 17:51:26.586268 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:26.585870    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-785dbf9c78-kqs5s" event={"ID":"e304865e-977a-4ca7-8518-4ddfd0849359","Type":"ContainerDied","Data":"07913e3aed65936529d296ea17e0c020904dc8e853c0bdce622994244c872ed1"}
Jul 26 17:51:26.586268 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:26.585888    2570 scope.go:117] "RemoveContainer" containerID="783a0eb94f9ba7861af0027a23d574c9c061ece24eee9ead27667df0c0f6e607"
Jul 26 17:51:26.594694 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:26.594680    2570 scope.go:117] "RemoveContainer" containerID="783a0eb94f9ba7861af0027a23d574c9c061ece24eee9ead27667df0c0f6e607"
Jul 26 17:51:26.594952 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:51:26.594933    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"783a0eb94f9ba7861af0027a23d574c9c061ece24eee9ead27667df0c0f6e607\": container with ID starting with 783a0eb94f9ba7861af0027a23d574c9c061ece24eee9ead27667df0c0f6e607 not found: ID does not exist" containerID="783a0eb94f9ba7861af0027a23d574c9c061ece24eee9ead27667df0c0f6e607"
Jul 26 17:51:26.595002 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:26.594960    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"783a0eb94f9ba7861af0027a23d574c9c061ece24eee9ead27667df0c0f6e607"} err="failed to get container status \"783a0eb94f9ba7861af0027a23d574c9c061ece24eee9ead27667df0c0f6e607\": rpc error: code = NotFound desc = could not find container \"783a0eb94f9ba7861af0027a23d574c9c061ece24eee9ead27667df0c0f6e607\": container with ID starting with 783a0eb94f9ba7861af0027a23d574c9c061ece24eee9ead27667df0c0f6e607 not found: ID does not exist"
Jul 26 17:51:26.605524 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:26.605501    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-785dbf9c78-kqs5s"]
Jul 26 17:51:26.611572 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:26.611551    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-785dbf9c78-kqs5s"]
Jul 26 17:51:27.837549 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:27.837515    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e304865e-977a-4ca7-8518-4ddfd0849359" path="/var/lib/kubelet/pods/e304865e-977a-4ca7-8518-4ddfd0849359/volumes"
Jul 26 17:51:49.759740 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.759697    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"]
Jul 26 17:51:49.760192 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.760095    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f532906-dddd-459d-b8da-3074eab679b1" containerName="registry"
Jul 26 17:51:49.760192 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.760127    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f532906-dddd-459d-b8da-3074eab679b1" containerName="registry"
Jul 26 17:51:49.760192 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.760153    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e304865e-977a-4ca7-8518-4ddfd0849359" containerName="console"
Jul 26 17:51:49.760192 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.760159    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e304865e-977a-4ca7-8518-4ddfd0849359" containerName="console"
Jul 26 17:51:49.760330 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.760204    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f532906-dddd-459d-b8da-3074eab679b1" containerName="registry"
Jul 26 17:51:49.760330 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.760214    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e304865e-977a-4ca7-8518-4ddfd0849359" containerName="console"
Jul 26 17:51:49.762705 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.762690    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:49.765046 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.765004    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-s8p4h\""
Jul 26 17:51:49.765260 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.765046    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 26 17:51:49.765260 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.765195    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 26 17:51:49.765436 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.765340    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 26 17:51:49.765570 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.765555    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 26 17:51:49.765695 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.765660    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 26 17:51:49.770615 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.770598    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 26 17:51:49.774984 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.774961    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"]
Jul 26 17:51:49.846254 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.846220    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/27d46ba6-398f-4d30-859d-b7a89e14942d-telemeter-client-tls\") pod \"telemeter-client-6db48c7d75-s7wz6\" (UID: \"27d46ba6-398f-4d30-859d-b7a89e14942d\") " pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:49.846409 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.846271    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27d46ba6-398f-4d30-859d-b7a89e14942d-metrics-client-ca\") pod \"telemeter-client-6db48c7d75-s7wz6\" (UID: \"27d46ba6-398f-4d30-859d-b7a89e14942d\") " pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:49.846409 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.846299    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/27d46ba6-398f-4d30-859d-b7a89e14942d-federate-client-tls\") pod \"telemeter-client-6db48c7d75-s7wz6\" (UID: \"27d46ba6-398f-4d30-859d-b7a89e14942d\") " pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:49.846409 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.846344    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhkjz\" (UniqueName: \"kubernetes.io/projected/27d46ba6-398f-4d30-859d-b7a89e14942d-kube-api-access-zhkjz\") pod \"telemeter-client-6db48c7d75-s7wz6\" (UID: \"27d46ba6-398f-4d30-859d-b7a89e14942d\") " pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:49.846409 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.846383    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27d46ba6-398f-4d30-859d-b7a89e14942d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6db48c7d75-s7wz6\" (UID: \"27d46ba6-398f-4d30-859d-b7a89e14942d\") " pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:49.846539 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.846417    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27d46ba6-398f-4d30-859d-b7a89e14942d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6db48c7d75-s7wz6\" (UID: \"27d46ba6-398f-4d30-859d-b7a89e14942d\") " pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:49.846539 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.846441    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27d46ba6-398f-4d30-859d-b7a89e14942d-serving-certs-ca-bundle\") pod \"telemeter-client-6db48c7d75-s7wz6\" (UID: \"27d46ba6-398f-4d30-859d-b7a89e14942d\") " pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:49.846539 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.846462    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/27d46ba6-398f-4d30-859d-b7a89e14942d-secret-telemeter-client\") pod \"telemeter-client-6db48c7d75-s7wz6\" (UID: \"27d46ba6-398f-4d30-859d-b7a89e14942d\") " pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:49.947419 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.947364    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/27d46ba6-398f-4d30-859d-b7a89e14942d-federate-client-tls\") pod \"telemeter-client-6db48c7d75-s7wz6\" (UID: \"27d46ba6-398f-4d30-859d-b7a89e14942d\") " pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:49.947419 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.947424    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhkjz\" (UniqueName: \"kubernetes.io/projected/27d46ba6-398f-4d30-859d-b7a89e14942d-kube-api-access-zhkjz\") pod \"telemeter-client-6db48c7d75-s7wz6\" (UID: \"27d46ba6-398f-4d30-859d-b7a89e14942d\") " pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:49.947644 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.947446    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27d46ba6-398f-4d30-859d-b7a89e14942d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6db48c7d75-s7wz6\" (UID: \"27d46ba6-398f-4d30-859d-b7a89e14942d\") " pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:49.947644 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.947478    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27d46ba6-398f-4d30-859d-b7a89e14942d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6db48c7d75-s7wz6\" (UID: \"27d46ba6-398f-4d30-859d-b7a89e14942d\") " pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:49.947644 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.947497    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27d46ba6-398f-4d30-859d-b7a89e14942d-serving-certs-ca-bundle\") pod \"telemeter-client-6db48c7d75-s7wz6\" (UID: \"27d46ba6-398f-4d30-859d-b7a89e14942d\") " pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:49.947644 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.947518    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/27d46ba6-398f-4d30-859d-b7a89e14942d-secret-telemeter-client\") pod \"telemeter-client-6db48c7d75-s7wz6\" (UID: \"27d46ba6-398f-4d30-859d-b7a89e14942d\") " pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:49.947644 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.947566    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/27d46ba6-398f-4d30-859d-b7a89e14942d-telemeter-client-tls\") pod \"telemeter-client-6db48c7d75-s7wz6\" (UID: \"27d46ba6-398f-4d30-859d-b7a89e14942d\") " pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:49.947644 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.947612    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27d46ba6-398f-4d30-859d-b7a89e14942d-metrics-client-ca\") pod \"telemeter-client-6db48c7d75-s7wz6\" (UID: \"27d46ba6-398f-4d30-859d-b7a89e14942d\") " pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:49.948470 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.948439    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27d46ba6-398f-4d30-859d-b7a89e14942d-serving-certs-ca-bundle\") pod \"telemeter-client-6db48c7d75-s7wz6\" (UID: \"27d46ba6-398f-4d30-859d-b7a89e14942d\") " pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:49.948611 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.948479    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27d46ba6-398f-4d30-859d-b7a89e14942d-metrics-client-ca\") pod \"telemeter-client-6db48c7d75-s7wz6\" (UID: \"27d46ba6-398f-4d30-859d-b7a89e14942d\") " pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:49.948611 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.948492    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27d46ba6-398f-4d30-859d-b7a89e14942d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6db48c7d75-s7wz6\" (UID: \"27d46ba6-398f-4d30-859d-b7a89e14942d\") " pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:49.950176 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.950152    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/27d46ba6-398f-4d30-859d-b7a89e14942d-secret-telemeter-client\") pod \"telemeter-client-6db48c7d75-s7wz6\" (UID: \"27d46ba6-398f-4d30-859d-b7a89e14942d\") " pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:49.950278 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.950209    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/27d46ba6-398f-4d30-859d-b7a89e14942d-federate-client-tls\") pod \"telemeter-client-6db48c7d75-s7wz6\" (UID: \"27d46ba6-398f-4d30-859d-b7a89e14942d\") " pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:49.950398 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.950382    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27d46ba6-398f-4d30-859d-b7a89e14942d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6db48c7d75-s7wz6\" (UID: \"27d46ba6-398f-4d30-859d-b7a89e14942d\") " pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:49.950583 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.950562    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/27d46ba6-398f-4d30-859d-b7a89e14942d-telemeter-client-tls\") pod \"telemeter-client-6db48c7d75-s7wz6\" (UID: \"27d46ba6-398f-4d30-859d-b7a89e14942d\") " pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:49.955925 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:49.955896    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhkjz\" (UniqueName: \"kubernetes.io/projected/27d46ba6-398f-4d30-859d-b7a89e14942d-kube-api-access-zhkjz\") pod \"telemeter-client-6db48c7d75-s7wz6\" (UID: \"27d46ba6-398f-4d30-859d-b7a89e14942d\") " pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:50.072716 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:50.072631    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"
Jul 26 17:51:50.200887 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:50.200853    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6db48c7d75-s7wz6"]
Jul 26 17:51:50.204855 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:51:50.204829    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27d46ba6_398f_4d30_859d_b7a89e14942d.slice/crio-e64e34c018662f131cda28bd7b328d41e4a3d287f4563e564c843a1209ee2e76 WatchSource:0}: Error finding container e64e34c018662f131cda28bd7b328d41e4a3d287f4563e564c843a1209ee2e76: Status 404 returned error can't find the container with id e64e34c018662f131cda28bd7b328d41e4a3d287f4563e564c843a1209ee2e76
Jul 26 17:51:50.674858 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:50.674820    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6" event={"ID":"27d46ba6-398f-4d30-859d-b7a89e14942d","Type":"ContainerStarted","Data":"e64e34c018662f131cda28bd7b328d41e4a3d287f4563e564c843a1209ee2e76"}
Jul 26 17:51:52.684161 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:52.684127    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6" event={"ID":"27d46ba6-398f-4d30-859d-b7a89e14942d","Type":"ContainerStarted","Data":"56d4654f3219ab70914793d5827634aa833ce894d45041d1e247403d50099ca8"}
Jul 26 17:51:53.689522 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:53.689487    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6" event={"ID":"27d46ba6-398f-4d30-859d-b7a89e14942d","Type":"ContainerStarted","Data":"fa98bd1fa50e98ac1e7e204e7f987f8ff3b41461f852f69a7e60bcb8b66c7bee"}
Jul 26 17:51:53.689898 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:53.689543    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6" event={"ID":"27d46ba6-398f-4d30-859d-b7a89e14942d","Type":"ContainerStarted","Data":"28737898d25f7ed75a4fd83fa7a3d159ca4fb9e03915d4db1f8d79b0c9d9985d"}
Jul 26 17:51:53.709479 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:53.709437    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6db48c7d75-s7wz6" podStartSLOduration=1.910776527 podStartE2EDuration="4.709421443s" totalImagesPullingTime="2.798644916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:51:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:51:53.708565983 +0000 UTC m=+170.360756841" watchObservedRunningTime="2026-07-26 17:51:53.709421443 +0000 UTC m=+170.361612298"
Jul 26 17:51:54.465561 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.465527    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-867b8cfb6d-9ntjs"]
Jul 26 17:51:54.467943 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.467910    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:51:54.479806 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.479784    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-867b8cfb6d-9ntjs"]
Jul 26 17:51:54.590488 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.590453    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2f763af3-5c50-42ae-9bf5-1d27a421c085-console-config\") pod \"console-867b8cfb6d-9ntjs\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") " pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:51:54.590658 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.590499    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2f763af3-5c50-42ae-9bf5-1d27a421c085-oauth-serving-cert\") pod \"console-867b8cfb6d-9ntjs\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") " pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:51:54.590658 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.590631    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f763af3-5c50-42ae-9bf5-1d27a421c085-console-serving-cert\") pod \"console-867b8cfb6d-9ntjs\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") " pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:51:54.590797 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.590677    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2f763af3-5c50-42ae-9bf5-1d27a421c085-service-ca\") pod \"console-867b8cfb6d-9ntjs\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") " pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:51:54.590797 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.590704    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2f763af3-5c50-42ae-9bf5-1d27a421c085-console-oauth-config\") pod \"console-867b8cfb6d-9ntjs\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") " pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:51:54.590797 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.590726    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f763af3-5c50-42ae-9bf5-1d27a421c085-trusted-ca-bundle\") pod \"console-867b8cfb6d-9ntjs\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") " pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:51:54.590797 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.590750    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2wzn\" (UniqueName: \"kubernetes.io/projected/2f763af3-5c50-42ae-9bf5-1d27a421c085-kube-api-access-b2wzn\") pod \"console-867b8cfb6d-9ntjs\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") " pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:51:54.691576 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.691535    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2f763af3-5c50-42ae-9bf5-1d27a421c085-service-ca\") pod \"console-867b8cfb6d-9ntjs\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") " pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:51:54.691576 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.691570    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2f763af3-5c50-42ae-9bf5-1d27a421c085-console-oauth-config\") pod \"console-867b8cfb6d-9ntjs\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") " pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:51:54.691576 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.691586    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f763af3-5c50-42ae-9bf5-1d27a421c085-trusted-ca-bundle\") pod \"console-867b8cfb6d-9ntjs\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") " pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:51:54.692221 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.691640    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2wzn\" (UniqueName: \"kubernetes.io/projected/2f763af3-5c50-42ae-9bf5-1d27a421c085-kube-api-access-b2wzn\") pod \"console-867b8cfb6d-9ntjs\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") " pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:51:54.692221 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.691707    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2f763af3-5c50-42ae-9bf5-1d27a421c085-console-config\") pod \"console-867b8cfb6d-9ntjs\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") " pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:51:54.692221 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.691765    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2f763af3-5c50-42ae-9bf5-1d27a421c085-oauth-serving-cert\") pod \"console-867b8cfb6d-9ntjs\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") " pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:51:54.692221 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.691844    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f763af3-5c50-42ae-9bf5-1d27a421c085-console-serving-cert\") pod \"console-867b8cfb6d-9ntjs\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") " pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:51:54.692571 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.692548    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f763af3-5c50-42ae-9bf5-1d27a421c085-trusted-ca-bundle\") pod \"console-867b8cfb6d-9ntjs\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") " pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:51:54.692682 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.692611    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2f763af3-5c50-42ae-9bf5-1d27a421c085-oauth-serving-cert\") pod \"console-867b8cfb6d-9ntjs\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") " pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:51:54.692682 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.692659    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2f763af3-5c50-42ae-9bf5-1d27a421c085-console-config\") pod \"console-867b8cfb6d-9ntjs\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") " pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:51:54.692931 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.692913    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2f763af3-5c50-42ae-9bf5-1d27a421c085-service-ca\") pod \"console-867b8cfb6d-9ntjs\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") " pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:51:54.694718 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.694696    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f763af3-5c50-42ae-9bf5-1d27a421c085-console-serving-cert\") pod \"console-867b8cfb6d-9ntjs\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") " pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:51:54.694807 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.694723    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2f763af3-5c50-42ae-9bf5-1d27a421c085-console-oauth-config\") pod \"console-867b8cfb6d-9ntjs\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") " pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:51:54.700352 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.700328    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2wzn\" (UniqueName: \"kubernetes.io/projected/2f763af3-5c50-42ae-9bf5-1d27a421c085-kube-api-access-b2wzn\") pod \"console-867b8cfb6d-9ntjs\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") " pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:51:54.779138 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.779072    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:51:54.909051 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:54.909028    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-867b8cfb6d-9ntjs"]
Jul 26 17:51:54.911478 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:51:54.911452    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f763af3_5c50_42ae_9bf5_1d27a421c085.slice/crio-9e00fbdeea0e9892ddd74a2f8f205d3e46c74a3e6403d2643658d26d5c18795f WatchSource:0}: Error finding container 9e00fbdeea0e9892ddd74a2f8f205d3e46c74a3e6403d2643658d26d5c18795f: Status 404 returned error can't find the container with id 9e00fbdeea0e9892ddd74a2f8f205d3e46c74a3e6403d2643658d26d5c18795f
Jul 26 17:51:55.700296 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:55.700254    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-867b8cfb6d-9ntjs" event={"ID":"2f763af3-5c50-42ae-9bf5-1d27a421c085","Type":"ContainerStarted","Data":"c21d18bb77ae88e4e904ffa189cd7faed9395f493ae6a8f5aeaec1bc68b5290b"}
Jul 26 17:51:55.700736 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:55.700307    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-867b8cfb6d-9ntjs" event={"ID":"2f763af3-5c50-42ae-9bf5-1d27a421c085","Type":"ContainerStarted","Data":"9e00fbdeea0e9892ddd74a2f8f205d3e46c74a3e6403d2643658d26d5c18795f"}
Jul 26 17:51:55.717739 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:51:55.717690    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-867b8cfb6d-9ntjs" podStartSLOduration=1.717674189 podStartE2EDuration="1.717674189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:51:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:51:55.716912592 +0000 UTC m=+172.369103450" watchObservedRunningTime="2026-07-26 17:51:55.717674189 +0000 UTC m=+172.369865045"
Jul 26 17:52:04.779664 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:04.779619    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:52:04.780031 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:04.779692    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:52:04.784527 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:04.784498    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:52:05.739225 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:05.739199    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:52:05.787186 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:05.787154    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b785b5d76-zcrkd"]
Jul 26 17:52:30.812093 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:30.812042    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5b785b5d76-zcrkd" podUID="472f1421-a4fb-4a72-a2f3-46b463b71e53" containerName="console" containerID="cri-o://1b68e3c1fc0a7998f7288e9655c626af9e9bb90b1098c38ae2c1795e83638122" gracePeriod=15
Jul 26 17:52:31.059840 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.059816    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:52:31.065713 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.065660    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/472f1421-a4fb-4a72-a2f3-46b463b71e53-trusted-ca-bundle\") pod \"472f1421-a4fb-4a72-a2f3-46b463b71e53\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") "
Jul 26 17:52:31.065713 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.065693    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/472f1421-a4fb-4a72-a2f3-46b463b71e53-console-serving-cert\") pod \"472f1421-a4fb-4a72-a2f3-46b463b71e53\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") "
Jul 26 17:52:31.065862 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.065737    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/472f1421-a4fb-4a72-a2f3-46b463b71e53-service-ca\") pod \"472f1421-a4fb-4a72-a2f3-46b463b71e53\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") "
Jul 26 17:52:31.065862 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.065784    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/472f1421-a4fb-4a72-a2f3-46b463b71e53-console-config\") pod \"472f1421-a4fb-4a72-a2f3-46b463b71e53\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") "
Jul 26 17:52:31.065862 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.065810    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/472f1421-a4fb-4a72-a2f3-46b463b71e53-console-oauth-config\") pod \"472f1421-a4fb-4a72-a2f3-46b463b71e53\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") "
Jul 26 17:52:31.065862 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.065850    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/472f1421-a4fb-4a72-a2f3-46b463b71e53-oauth-serving-cert\") pod \"472f1421-a4fb-4a72-a2f3-46b463b71e53\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") "
Jul 26 17:52:31.066053 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.065879    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4qv8\" (UniqueName: \"kubernetes.io/projected/472f1421-a4fb-4a72-a2f3-46b463b71e53-kube-api-access-p4qv8\") pod \"472f1421-a4fb-4a72-a2f3-46b463b71e53\" (UID: \"472f1421-a4fb-4a72-a2f3-46b463b71e53\") "
Jul 26 17:52:31.066154 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.066128    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/472f1421-a4fb-4a72-a2f3-46b463b71e53-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "472f1421-a4fb-4a72-a2f3-46b463b71e53" (UID: "472f1421-a4fb-4a72-a2f3-46b463b71e53"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:52:31.066273 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.066252    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/472f1421-a4fb-4a72-a2f3-46b463b71e53-service-ca" (OuterVolumeSpecName: "service-ca") pod "472f1421-a4fb-4a72-a2f3-46b463b71e53" (UID: "472f1421-a4fb-4a72-a2f3-46b463b71e53"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:52:31.066273 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.066258    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/472f1421-a4fb-4a72-a2f3-46b463b71e53-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "472f1421-a4fb-4a72-a2f3-46b463b71e53" (UID: "472f1421-a4fb-4a72-a2f3-46b463b71e53"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:52:31.066363 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.066295    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/472f1421-a4fb-4a72-a2f3-46b463b71e53-console-config" (OuterVolumeSpecName: "console-config") pod "472f1421-a4fb-4a72-a2f3-46b463b71e53" (UID: "472f1421-a4fb-4a72-a2f3-46b463b71e53"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:52:31.067806 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.067782    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/472f1421-a4fb-4a72-a2f3-46b463b71e53-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "472f1421-a4fb-4a72-a2f3-46b463b71e53" (UID: "472f1421-a4fb-4a72-a2f3-46b463b71e53"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 17:52:31.067948 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.067926    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/472f1421-a4fb-4a72-a2f3-46b463b71e53-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "472f1421-a4fb-4a72-a2f3-46b463b71e53" (UID: "472f1421-a4fb-4a72-a2f3-46b463b71e53"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 17:52:31.067948 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.067938    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/472f1421-a4fb-4a72-a2f3-46b463b71e53-kube-api-access-p4qv8" (OuterVolumeSpecName: "kube-api-access-p4qv8") pod "472f1421-a4fb-4a72-a2f3-46b463b71e53" (UID: "472f1421-a4fb-4a72-a2f3-46b463b71e53"). InnerVolumeSpecName "kube-api-access-p4qv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:52:31.166616 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.166582    2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/472f1421-a4fb-4a72-a2f3-46b463b71e53-console-serving-cert\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:52:31.166616 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.166610    2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/472f1421-a4fb-4a72-a2f3-46b463b71e53-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:52:31.166616 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.166621    2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/472f1421-a4fb-4a72-a2f3-46b463b71e53-console-config\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:52:31.166846 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.166629    2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/472f1421-a4fb-4a72-a2f3-46b463b71e53-console-oauth-config\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:52:31.166846 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.166638    2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/472f1421-a4fb-4a72-a2f3-46b463b71e53-oauth-serving-cert\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:52:31.166846 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.166650    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4qv8\" (UniqueName: \"kubernetes.io/projected/472f1421-a4fb-4a72-a2f3-46b463b71e53-kube-api-access-p4qv8\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:52:31.166846 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.166658    2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/472f1421-a4fb-4a72-a2f3-46b463b71e53-trusted-ca-bundle\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:52:31.828571 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.828535    2570 generic.go:356] "Generic (PLEG): container finished" podID="472f1421-a4fb-4a72-a2f3-46b463b71e53" containerID="1b68e3c1fc0a7998f7288e9655c626af9e9bb90b1098c38ae2c1795e83638122" exitCode=0
Jul 26 17:52:31.828996 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.828583    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b785b5d76-zcrkd"
Jul 26 17:52:31.828996 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.828596    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b785b5d76-zcrkd" event={"ID":"472f1421-a4fb-4a72-a2f3-46b463b71e53","Type":"ContainerDied","Data":"1b68e3c1fc0a7998f7288e9655c626af9e9bb90b1098c38ae2c1795e83638122"}
Jul 26 17:52:31.828996 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.828666    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b785b5d76-zcrkd" event={"ID":"472f1421-a4fb-4a72-a2f3-46b463b71e53","Type":"ContainerDied","Data":"06e4fed72e559144754ca5d735079b91628e78f76ea3410df187eb99d6e9148f"}
Jul 26 17:52:31.828996 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.828688    2570 scope.go:117] "RemoveContainer" containerID="1b68e3c1fc0a7998f7288e9655c626af9e9bb90b1098c38ae2c1795e83638122"
Jul 26 17:52:31.837321 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.837159    2570 scope.go:117] "RemoveContainer" containerID="1b68e3c1fc0a7998f7288e9655c626af9e9bb90b1098c38ae2c1795e83638122"
Jul 26 17:52:31.837459 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:52:31.837434    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b68e3c1fc0a7998f7288e9655c626af9e9bb90b1098c38ae2c1795e83638122\": container with ID starting with 1b68e3c1fc0a7998f7288e9655c626af9e9bb90b1098c38ae2c1795e83638122 not found: ID does not exist" containerID="1b68e3c1fc0a7998f7288e9655c626af9e9bb90b1098c38ae2c1795e83638122"
Jul 26 17:52:31.837510 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.837473    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b68e3c1fc0a7998f7288e9655c626af9e9bb90b1098c38ae2c1795e83638122"} err="failed to get container status \"1b68e3c1fc0a7998f7288e9655c626af9e9bb90b1098c38ae2c1795e83638122\": rpc error: code = NotFound desc = could not find container \"1b68e3c1fc0a7998f7288e9655c626af9e9bb90b1098c38ae2c1795e83638122\": container with ID starting with 1b68e3c1fc0a7998f7288e9655c626af9e9bb90b1098c38ae2c1795e83638122 not found: ID does not exist"
Jul 26 17:52:31.850406 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.850373    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b785b5d76-zcrkd"]
Jul 26 17:52:31.854003 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:31.853974    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b785b5d76-zcrkd"]
Jul 26 17:52:33.837225 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:52:33.837193    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="472f1421-a4fb-4a72-a2f3-46b463b71e53" path="/var/lib/kubelet/pods/472f1421-a4fb-4a72-a2f3-46b463b71e53/volumes"
Jul 26 17:53:04.189552 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.189476    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76d9746fc5-hkp5d"]
Jul 26 17:53:04.189952 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.189776    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="472f1421-a4fb-4a72-a2f3-46b463b71e53" containerName="console"
Jul 26 17:53:04.189952 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.189793    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="472f1421-a4fb-4a72-a2f3-46b463b71e53" containerName="console"
Jul 26 17:53:04.189952 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.189857    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="472f1421-a4fb-4a72-a2f3-46b463b71e53" containerName="console"
Jul 26 17:53:04.194144 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.194096    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:04.203532 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.203504    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76d9746fc5-hkp5d"]
Jul 26 17:53:04.317412 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.317365    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01f883ba-c6d8-4071-8fe9-8f732fdb562f-trusted-ca-bundle\") pod \"console-76d9746fc5-hkp5d\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") " pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:04.317412 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.317412    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01f883ba-c6d8-4071-8fe9-8f732fdb562f-service-ca\") pod \"console-76d9746fc5-hkp5d\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") " pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:04.317662 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.317457    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01f883ba-c6d8-4071-8fe9-8f732fdb562f-console-config\") pod \"console-76d9746fc5-hkp5d\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") " pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:04.317662 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.317506    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01f883ba-c6d8-4071-8fe9-8f732fdb562f-oauth-serving-cert\") pod \"console-76d9746fc5-hkp5d\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") " pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:04.317662 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.317528    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01f883ba-c6d8-4071-8fe9-8f732fdb562f-console-oauth-config\") pod \"console-76d9746fc5-hkp5d\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") " pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:04.317662 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.317550    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01f883ba-c6d8-4071-8fe9-8f732fdb562f-console-serving-cert\") pod \"console-76d9746fc5-hkp5d\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") " pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:04.317662 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.317579    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx5qn\" (UniqueName: \"kubernetes.io/projected/01f883ba-c6d8-4071-8fe9-8f732fdb562f-kube-api-access-hx5qn\") pod \"console-76d9746fc5-hkp5d\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") " pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:04.418796 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.418740    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01f883ba-c6d8-4071-8fe9-8f732fdb562f-trusted-ca-bundle\") pod \"console-76d9746fc5-hkp5d\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") " pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:04.418796 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.418795    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01f883ba-c6d8-4071-8fe9-8f732fdb562f-service-ca\") pod \"console-76d9746fc5-hkp5d\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") " pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:04.419019 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.418929    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01f883ba-c6d8-4071-8fe9-8f732fdb562f-console-config\") pod \"console-76d9746fc5-hkp5d\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") " pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:04.419019 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.418983    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01f883ba-c6d8-4071-8fe9-8f732fdb562f-oauth-serving-cert\") pod \"console-76d9746fc5-hkp5d\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") " pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:04.419019 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.419008    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01f883ba-c6d8-4071-8fe9-8f732fdb562f-console-oauth-config\") pod \"console-76d9746fc5-hkp5d\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") " pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:04.419193 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.419046    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01f883ba-c6d8-4071-8fe9-8f732fdb562f-console-serving-cert\") pod \"console-76d9746fc5-hkp5d\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") " pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:04.419193 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.419087    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx5qn\" (UniqueName: \"kubernetes.io/projected/01f883ba-c6d8-4071-8fe9-8f732fdb562f-kube-api-access-hx5qn\") pod \"console-76d9746fc5-hkp5d\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") " pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:04.419636 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.419612    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01f883ba-c6d8-4071-8fe9-8f732fdb562f-console-config\") pod \"console-76d9746fc5-hkp5d\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") " pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:04.419744 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.419641    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01f883ba-c6d8-4071-8fe9-8f732fdb562f-service-ca\") pod \"console-76d9746fc5-hkp5d\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") " pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:04.419744 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.419685    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01f883ba-c6d8-4071-8fe9-8f732fdb562f-trusted-ca-bundle\") pod \"console-76d9746fc5-hkp5d\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") " pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:04.419744 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.419723    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01f883ba-c6d8-4071-8fe9-8f732fdb562f-oauth-serving-cert\") pod \"console-76d9746fc5-hkp5d\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") " pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:04.421649 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.421620    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01f883ba-c6d8-4071-8fe9-8f732fdb562f-console-oauth-config\") pod \"console-76d9746fc5-hkp5d\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") " pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:04.421736 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.421653    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01f883ba-c6d8-4071-8fe9-8f732fdb562f-console-serving-cert\") pod \"console-76d9746fc5-hkp5d\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") " pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:04.426832 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.426809    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx5qn\" (UniqueName: \"kubernetes.io/projected/01f883ba-c6d8-4071-8fe9-8f732fdb562f-kube-api-access-hx5qn\") pod \"console-76d9746fc5-hkp5d\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") " pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:04.504982 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.504875    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:04.634545 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.634499    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76d9746fc5-hkp5d"]
Jul 26 17:53:04.638702 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:53:04.638678    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01f883ba_c6d8_4071_8fe9_8f732fdb562f.slice/crio-c91ae0be424fb90d4be1721f97ef883d5374a9d67f03ec77a8ef60cb43b8b113 WatchSource:0}: Error finding container c91ae0be424fb90d4be1721f97ef883d5374a9d67f03ec77a8ef60cb43b8b113: Status 404 returned error can't find the container with id c91ae0be424fb90d4be1721f97ef883d5374a9d67f03ec77a8ef60cb43b8b113
Jul 26 17:53:04.947026 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.946989    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76d9746fc5-hkp5d" event={"ID":"01f883ba-c6d8-4071-8fe9-8f732fdb562f","Type":"ContainerStarted","Data":"665e661e2ecc74bd9a03150d1a3503c00f2dbafa1797c11a0505e175677ace9d"}
Jul 26 17:53:04.947254 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.947047    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76d9746fc5-hkp5d" event={"ID":"01f883ba-c6d8-4071-8fe9-8f732fdb562f","Type":"ContainerStarted","Data":"c91ae0be424fb90d4be1721f97ef883d5374a9d67f03ec77a8ef60cb43b8b113"}
Jul 26 17:53:04.966412 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:04.966364    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76d9746fc5-hkp5d" podStartSLOduration=0.966348897 podStartE2EDuration="966.348897ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:53:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:53:04.964609279 +0000 UTC m=+241.616800150" watchObservedRunningTime="2026-07-26 17:53:04.966348897 +0000 UTC m=+241.618539754"
Jul 26 17:53:14.505052 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:14.505013    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:14.505550 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:14.505078    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:14.509873 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:14.509853    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:14.987809 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:14.987781    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:53:15.033023 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:15.032987    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-867b8cfb6d-9ntjs"]
Jul 26 17:53:40.055430 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:40.055342    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-867b8cfb6d-9ntjs" podUID="2f763af3-5c50-42ae-9bf5-1d27a421c085" containerName="console" containerID="cri-o://c21d18bb77ae88e4e904ffa189cd7faed9395f493ae6a8f5aeaec1bc68b5290b" gracePeriod=15
Jul 26 17:53:40.296821 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:40.296798    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:53:40.312326 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:40.312277    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2f763af3-5c50-42ae-9bf5-1d27a421c085-oauth-serving-cert\") pod \"2f763af3-5c50-42ae-9bf5-1d27a421c085\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") "
Jul 26 17:53:40.312326 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:40.312308    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2f763af3-5c50-42ae-9bf5-1d27a421c085-console-config\") pod \"2f763af3-5c50-42ae-9bf5-1d27a421c085\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") "
Jul 26 17:53:40.312428 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:40.312327    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2f763af3-5c50-42ae-9bf5-1d27a421c085-service-ca\") pod \"2f763af3-5c50-42ae-9bf5-1d27a421c085\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") "
Jul 26 17:53:40.312428 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:40.312388    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2f763af3-5c50-42ae-9bf5-1d27a421c085-console-oauth-config\") pod \"2f763af3-5c50-42ae-9bf5-1d27a421c085\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") "
Jul 26 17:53:40.312428 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:40.312419    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2wzn\" (UniqueName: \"kubernetes.io/projected/2f763af3-5c50-42ae-9bf5-1d27a421c085-kube-api-access-b2wzn\") pod \"2f763af3-5c50-42ae-9bf5-1d27a421c085\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") "
Jul 26 17:53:40.312554 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:40.312450    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f763af3-5c50-42ae-9bf5-1d27a421c085-trusted-ca-bundle\") pod \"2f763af3-5c50-42ae-9bf5-1d27a421c085\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") "
Jul 26 17:53:40.312608 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:40.312547    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f763af3-5c50-42ae-9bf5-1d27a421c085-console-serving-cert\") pod \"2f763af3-5c50-42ae-9bf5-1d27a421c085\" (UID: \"2f763af3-5c50-42ae-9bf5-1d27a421c085\") "
Jul 26 17:53:40.312812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:40.312772    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f763af3-5c50-42ae-9bf5-1d27a421c085-console-config" (OuterVolumeSpecName: "console-config") pod "2f763af3-5c50-42ae-9bf5-1d27a421c085" (UID: "2f763af3-5c50-42ae-9bf5-1d27a421c085"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:53:40.312812 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:40.312802    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f763af3-5c50-42ae-9bf5-1d27a421c085-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2f763af3-5c50-42ae-9bf5-1d27a421c085" (UID: "2f763af3-5c50-42ae-9bf5-1d27a421c085"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:53:40.312917 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:40.312902    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f763af3-5c50-42ae-9bf5-1d27a421c085-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2f763af3-5c50-42ae-9bf5-1d27a421c085" (UID: "2f763af3-5c50-42ae-9bf5-1d27a421c085"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:53:40.312953 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:40.312915    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f763af3-5c50-42ae-9bf5-1d27a421c085-service-ca" (OuterVolumeSpecName: "service-ca") pod "2f763af3-5c50-42ae-9bf5-1d27a421c085" (UID: "2f763af3-5c50-42ae-9bf5-1d27a421c085"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:53:40.314836 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:40.314807    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f763af3-5c50-42ae-9bf5-1d27a421c085-kube-api-access-b2wzn" (OuterVolumeSpecName: "kube-api-access-b2wzn") pod "2f763af3-5c50-42ae-9bf5-1d27a421c085" (UID: "2f763af3-5c50-42ae-9bf5-1d27a421c085"). InnerVolumeSpecName "kube-api-access-b2wzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:53:40.314956 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:40.314890    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f763af3-5c50-42ae-9bf5-1d27a421c085-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2f763af3-5c50-42ae-9bf5-1d27a421c085" (UID: "2f763af3-5c50-42ae-9bf5-1d27a421c085"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 17:53:40.315014 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:40.315000    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f763af3-5c50-42ae-9bf5-1d27a421c085-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2f763af3-5c50-42ae-9bf5-1d27a421c085" (UID: "2f763af3-5c50-42ae-9bf5-1d27a421c085"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 17:53:40.413249 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:40.413203    2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f763af3-5c50-42ae-9bf5-1d27a421c085-console-serving-cert\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:53:40.413249 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:40.413239    2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2f763af3-5c50-42ae-9bf5-1d27a421c085-oauth-serving-cert\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:53:40.413249 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:40.413250    2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2f763af3-5c50-42ae-9bf5-1d27a421c085-console-config\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:53:40.413249 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:40.413259    2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2f763af3-5c50-42ae-9bf5-1d27a421c085-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:53:40.413526 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:40.413267    2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2f763af3-5c50-42ae-9bf5-1d27a421c085-console-oauth-config\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:53:40.413526 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:40.413277    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2wzn\" (UniqueName: \"kubernetes.io/projected/2f763af3-5c50-42ae-9bf5-1d27a421c085-kube-api-access-b2wzn\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:53:40.413526 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:40.413285    2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f763af3-5c50-42ae-9bf5-1d27a421c085-trusted-ca-bundle\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:53:41.076979 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:41.076945    2570 generic.go:356] "Generic (PLEG): container finished" podID="2f763af3-5c50-42ae-9bf5-1d27a421c085" containerID="c21d18bb77ae88e4e904ffa189cd7faed9395f493ae6a8f5aeaec1bc68b5290b" exitCode=0
Jul 26 17:53:41.077418 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:41.077002    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-867b8cfb6d-9ntjs"
Jul 26 17:53:41.077418 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:41.077031    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-867b8cfb6d-9ntjs" event={"ID":"2f763af3-5c50-42ae-9bf5-1d27a421c085","Type":"ContainerDied","Data":"c21d18bb77ae88e4e904ffa189cd7faed9395f493ae6a8f5aeaec1bc68b5290b"}
Jul 26 17:53:41.077418 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:41.077082    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-867b8cfb6d-9ntjs" event={"ID":"2f763af3-5c50-42ae-9bf5-1d27a421c085","Type":"ContainerDied","Data":"9e00fbdeea0e9892ddd74a2f8f205d3e46c74a3e6403d2643658d26d5c18795f"}
Jul 26 17:53:41.077418 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:41.077104    2570 scope.go:117] "RemoveContainer" containerID="c21d18bb77ae88e4e904ffa189cd7faed9395f493ae6a8f5aeaec1bc68b5290b"
Jul 26 17:53:41.085540 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:41.085523    2570 scope.go:117] "RemoveContainer" containerID="c21d18bb77ae88e4e904ffa189cd7faed9395f493ae6a8f5aeaec1bc68b5290b"
Jul 26 17:53:41.085780 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:53:41.085764    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c21d18bb77ae88e4e904ffa189cd7faed9395f493ae6a8f5aeaec1bc68b5290b\": container with ID starting with c21d18bb77ae88e4e904ffa189cd7faed9395f493ae6a8f5aeaec1bc68b5290b not found: ID does not exist" containerID="c21d18bb77ae88e4e904ffa189cd7faed9395f493ae6a8f5aeaec1bc68b5290b"
Jul 26 17:53:41.085824 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:41.085789    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c21d18bb77ae88e4e904ffa189cd7faed9395f493ae6a8f5aeaec1bc68b5290b"} err="failed to get container status \"c21d18bb77ae88e4e904ffa189cd7faed9395f493ae6a8f5aeaec1bc68b5290b\": rpc error: code = NotFound desc = could not find container \"c21d18bb77ae88e4e904ffa189cd7faed9395f493ae6a8f5aeaec1bc68b5290b\": container with ID starting with c21d18bb77ae88e4e904ffa189cd7faed9395f493ae6a8f5aeaec1bc68b5290b not found: ID does not exist"
Jul 26 17:53:41.098854 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:41.098830    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-867b8cfb6d-9ntjs"]
Jul 26 17:53:41.102856 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:41.102837    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-867b8cfb6d-9ntjs"]
Jul 26 17:53:41.837015 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:53:41.836985    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f763af3-5c50-42ae-9bf5-1d27a421c085" path="/var/lib/kubelet/pods/2f763af3-5c50-42ae-9bf5-1d27a421c085/volumes"
Jul 26 17:54:03.768161 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:03.768131    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 17:54:03.775127 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:03.775089    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 17:54:03.777134 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:03.777099    2570 kubelet.go:1648] "Image garbage collection succeeded"
Jul 26 17:54:08.135479 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:08.135439    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv"]
Jul 26 17:54:08.137977 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:08.135714    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f763af3-5c50-42ae-9bf5-1d27a421c085" containerName="console"
Jul 26 17:54:08.137977 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:08.135725    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f763af3-5c50-42ae-9bf5-1d27a421c085" containerName="console"
Jul 26 17:54:08.137977 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:08.135800    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f763af3-5c50-42ae-9bf5-1d27a421c085" containerName="console"
Jul 26 17:54:08.138751 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:08.138734    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv"
Jul 26 17:54:08.141166 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:08.141144    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 26 17:54:08.141281 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:08.141148    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-b628c\""
Jul 26 17:54:08.141622 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:08.141604    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 26 17:54:08.146930 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:08.146910    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv"]
Jul 26 17:54:08.219585 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:08.219547    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2da5b0b5-fcdc-49d9-b904-72844d3e24a3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv\" (UID: \"2da5b0b5-fcdc-49d9-b904-72844d3e24a3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv"
Jul 26 17:54:08.219755 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:08.219597    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n982\" (UniqueName: \"kubernetes.io/projected/2da5b0b5-fcdc-49d9-b904-72844d3e24a3-kube-api-access-8n982\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv\" (UID: \"2da5b0b5-fcdc-49d9-b904-72844d3e24a3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv"
Jul 26 17:54:08.219755 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:08.219650    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2da5b0b5-fcdc-49d9-b904-72844d3e24a3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv\" (UID: \"2da5b0b5-fcdc-49d9-b904-72844d3e24a3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv"
Jul 26 17:54:08.320137 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:08.320084    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2da5b0b5-fcdc-49d9-b904-72844d3e24a3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv\" (UID: \"2da5b0b5-fcdc-49d9-b904-72844d3e24a3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv"
Jul 26 17:54:08.320297 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:08.320168    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2da5b0b5-fcdc-49d9-b904-72844d3e24a3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv\" (UID: \"2da5b0b5-fcdc-49d9-b904-72844d3e24a3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv"
Jul 26 17:54:08.320297 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:08.320196    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n982\" (UniqueName: \"kubernetes.io/projected/2da5b0b5-fcdc-49d9-b904-72844d3e24a3-kube-api-access-8n982\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv\" (UID: \"2da5b0b5-fcdc-49d9-b904-72844d3e24a3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv"
Jul 26 17:54:08.320581 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:08.320558    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2da5b0b5-fcdc-49d9-b904-72844d3e24a3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv\" (UID: \"2da5b0b5-fcdc-49d9-b904-72844d3e24a3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv"
Jul 26 17:54:08.320617 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:08.320565    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2da5b0b5-fcdc-49d9-b904-72844d3e24a3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv\" (UID: \"2da5b0b5-fcdc-49d9-b904-72844d3e24a3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv"
Jul 26 17:54:08.328448 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:08.328417    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n982\" (UniqueName: \"kubernetes.io/projected/2da5b0b5-fcdc-49d9-b904-72844d3e24a3-kube-api-access-8n982\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv\" (UID: \"2da5b0b5-fcdc-49d9-b904-72844d3e24a3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv"
Jul 26 17:54:08.448695 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:08.448609    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv"
Jul 26 17:54:08.574590 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:08.574555    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv"]
Jul 26 17:54:08.577878 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:54:08.577852    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2da5b0b5_fcdc_49d9_b904_72844d3e24a3.slice/crio-c344ff7785bb582723efeb8ec661669b80d1c0c804c7811288504825adc318bc WatchSource:0}: Error finding container c344ff7785bb582723efeb8ec661669b80d1c0c804c7811288504825adc318bc: Status 404 returned error can't find the container with id c344ff7785bb582723efeb8ec661669b80d1c0c804c7811288504825adc318bc
Jul 26 17:54:08.580567 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:08.580550    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 17:54:09.179203 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:09.179168    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv" event={"ID":"2da5b0b5-fcdc-49d9-b904-72844d3e24a3","Type":"ContainerStarted","Data":"c344ff7785bb582723efeb8ec661669b80d1c0c804c7811288504825adc318bc"}
Jul 26 17:54:15.206407 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:15.206312    2570 generic.go:356] "Generic (PLEG): container finished" podID="2da5b0b5-fcdc-49d9-b904-72844d3e24a3" containerID="f8e3b00a1335aade929d6a43175e8061445c7f8039bb0cebe2977d1f799a5cf4" exitCode=0
Jul 26 17:54:15.206407 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:15.206383    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv" event={"ID":"2da5b0b5-fcdc-49d9-b904-72844d3e24a3","Type":"ContainerDied","Data":"f8e3b00a1335aade929d6a43175e8061445c7f8039bb0cebe2977d1f799a5cf4"}
Jul 26 17:54:18.220132 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:18.220081    2570 generic.go:356] "Generic (PLEG): container finished" podID="2da5b0b5-fcdc-49d9-b904-72844d3e24a3" containerID="06202eea474b62b0f60d26a1269d639828628148dfee14864adb8f436250c339" exitCode=0
Jul 26 17:54:18.220487 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:18.220172    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv" event={"ID":"2da5b0b5-fcdc-49d9-b904-72844d3e24a3","Type":"ContainerDied","Data":"06202eea474b62b0f60d26a1269d639828628148dfee14864adb8f436250c339"}
Jul 26 17:54:25.251790 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:25.251696    2570 generic.go:356] "Generic (PLEG): container finished" podID="2da5b0b5-fcdc-49d9-b904-72844d3e24a3" containerID="6ba3de2d3bc241a7106d338145474f2c57664c199b519e455c99d24418abe57c" exitCode=0
Jul 26 17:54:25.251790 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:25.251772    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv" event={"ID":"2da5b0b5-fcdc-49d9-b904-72844d3e24a3","Type":"ContainerDied","Data":"6ba3de2d3bc241a7106d338145474f2c57664c199b519e455c99d24418abe57c"}
Jul 26 17:54:26.375555 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:26.375530    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv"
Jul 26 17:54:26.478183 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:26.478155    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2da5b0b5-fcdc-49d9-b904-72844d3e24a3-util\") pod \"2da5b0b5-fcdc-49d9-b904-72844d3e24a3\" (UID: \"2da5b0b5-fcdc-49d9-b904-72844d3e24a3\") "
Jul 26 17:54:26.478366 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:26.478206    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8n982\" (UniqueName: \"kubernetes.io/projected/2da5b0b5-fcdc-49d9-b904-72844d3e24a3-kube-api-access-8n982\") pod \"2da5b0b5-fcdc-49d9-b904-72844d3e24a3\" (UID: \"2da5b0b5-fcdc-49d9-b904-72844d3e24a3\") "
Jul 26 17:54:26.478366 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:26.478245    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2da5b0b5-fcdc-49d9-b904-72844d3e24a3-bundle\") pod \"2da5b0b5-fcdc-49d9-b904-72844d3e24a3\" (UID: \"2da5b0b5-fcdc-49d9-b904-72844d3e24a3\") "
Jul 26 17:54:26.478957 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:26.478934    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2da5b0b5-fcdc-49d9-b904-72844d3e24a3-bundle" (OuterVolumeSpecName: "bundle") pod "2da5b0b5-fcdc-49d9-b904-72844d3e24a3" (UID: "2da5b0b5-fcdc-49d9-b904-72844d3e24a3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 17:54:26.480464 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:26.480443    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2da5b0b5-fcdc-49d9-b904-72844d3e24a3-kube-api-access-8n982" (OuterVolumeSpecName: "kube-api-access-8n982") pod "2da5b0b5-fcdc-49d9-b904-72844d3e24a3" (UID: "2da5b0b5-fcdc-49d9-b904-72844d3e24a3"). InnerVolumeSpecName "kube-api-access-8n982". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:54:26.483144 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:26.483102    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2da5b0b5-fcdc-49d9-b904-72844d3e24a3-util" (OuterVolumeSpecName: "util") pod "2da5b0b5-fcdc-49d9-b904-72844d3e24a3" (UID: "2da5b0b5-fcdc-49d9-b904-72844d3e24a3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 17:54:26.579751 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:26.579714    2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2da5b0b5-fcdc-49d9-b904-72844d3e24a3-bundle\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:54:26.579751 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:26.579749    2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2da5b0b5-fcdc-49d9-b904-72844d3e24a3-util\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:54:26.579950 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:26.579761    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8n982\" (UniqueName: \"kubernetes.io/projected/2da5b0b5-fcdc-49d9-b904-72844d3e24a3-kube-api-access-8n982\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:54:27.260902 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:27.260867    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv" event={"ID":"2da5b0b5-fcdc-49d9-b904-72844d3e24a3","Type":"ContainerDied","Data":"c344ff7785bb582723efeb8ec661669b80d1c0c804c7811288504825adc318bc"}
Jul 26 17:54:27.261077 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:27.260940    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c344ff7785bb582723efeb8ec661669b80d1c0c804c7811288504825adc318bc"
Jul 26 17:54:27.261077 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:27.260875    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gfsgv"
Jul 26 17:54:31.067088 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:31.067051    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lktxk"]
Jul 26 17:54:31.067483 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:31.067338    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2da5b0b5-fcdc-49d9-b904-72844d3e24a3" containerName="util"
Jul 26 17:54:31.067483 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:31.067349    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2da5b0b5-fcdc-49d9-b904-72844d3e24a3" containerName="util"
Jul 26 17:54:31.067483 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:31.067366    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2da5b0b5-fcdc-49d9-b904-72844d3e24a3" containerName="extract"
Jul 26 17:54:31.067483 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:31.067372    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2da5b0b5-fcdc-49d9-b904-72844d3e24a3" containerName="extract"
Jul 26 17:54:31.067483 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:31.067383    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2da5b0b5-fcdc-49d9-b904-72844d3e24a3" containerName="pull"
Jul 26 17:54:31.067483 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:31.067388    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2da5b0b5-fcdc-49d9-b904-72844d3e24a3" containerName="pull"
Jul 26 17:54:31.067483 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:31.067431    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2da5b0b5-fcdc-49d9-b904-72844d3e24a3" containerName="extract"
Jul 26 17:54:31.070544 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:31.070528    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lktxk"
Jul 26 17:54:31.072763 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:31.072741    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-v8kr9\""
Jul 26 17:54:31.072896 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:31.072777    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 26 17:54:31.073128 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:31.073097    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 26 17:54:31.086260 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:31.086234    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lktxk"]
Jul 26 17:54:31.216895 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:31.216855    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkgsd\" (UniqueName: \"kubernetes.io/projected/2ddcd619-29ae-460b-b304-86f67aabdaa1-kube-api-access-kkgsd\") pod \"cert-manager-operator-controller-manager-55c444bf86-lktxk\" (UID: \"2ddcd619-29ae-460b-b304-86f67aabdaa1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lktxk"
Jul 26 17:54:31.216895 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:31.216898    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ddcd619-29ae-460b-b304-86f67aabdaa1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lktxk\" (UID: \"2ddcd619-29ae-460b-b304-86f67aabdaa1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lktxk"
Jul 26 17:54:31.318405 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:31.318317    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkgsd\" (UniqueName: \"kubernetes.io/projected/2ddcd619-29ae-460b-b304-86f67aabdaa1-kube-api-access-kkgsd\") pod \"cert-manager-operator-controller-manager-55c444bf86-lktxk\" (UID: \"2ddcd619-29ae-460b-b304-86f67aabdaa1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lktxk"
Jul 26 17:54:31.318405 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:31.318349    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ddcd619-29ae-460b-b304-86f67aabdaa1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lktxk\" (UID: \"2ddcd619-29ae-460b-b304-86f67aabdaa1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lktxk"
Jul 26 17:54:31.318675 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:31.318658    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ddcd619-29ae-460b-b304-86f67aabdaa1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lktxk\" (UID: \"2ddcd619-29ae-460b-b304-86f67aabdaa1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lktxk"
Jul 26 17:54:31.327279 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:31.327251    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkgsd\" (UniqueName: \"kubernetes.io/projected/2ddcd619-29ae-460b-b304-86f67aabdaa1-kube-api-access-kkgsd\") pod \"cert-manager-operator-controller-manager-55c444bf86-lktxk\" (UID: \"2ddcd619-29ae-460b-b304-86f67aabdaa1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lktxk"
Jul 26 17:54:31.379849 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:31.379816    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lktxk"
Jul 26 17:54:31.513047 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:31.513002    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lktxk"]
Jul 26 17:54:31.515301 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:54:31.515270    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ddcd619_29ae_460b_b304_86f67aabdaa1.slice/crio-913670b7151f63cbbf204ffbc9668853948258789e97c137d83c8ea87f0dff85 WatchSource:0}: Error finding container 913670b7151f63cbbf204ffbc9668853948258789e97c137d83c8ea87f0dff85: Status 404 returned error can't find the container with id 913670b7151f63cbbf204ffbc9668853948258789e97c137d83c8ea87f0dff85
Jul 26 17:54:32.282370 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:32.282331    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lktxk" event={"ID":"2ddcd619-29ae-460b-b304-86f67aabdaa1","Type":"ContainerStarted","Data":"913670b7151f63cbbf204ffbc9668853948258789e97c137d83c8ea87f0dff85"}
Jul 26 17:54:33.289185 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:33.289069    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lktxk" event={"ID":"2ddcd619-29ae-460b-b304-86f67aabdaa1","Type":"ContainerStarted","Data":"43bbccf75b4630b6cb452cdd82814b285495b1ee7edf6d5d0a8384b421be5b5b"}
Jul 26 17:54:33.314943 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:33.314888    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lktxk" podStartSLOduration=0.662194801 podStartE2EDuration="2.314872225s" totalImagesPullingTime="1.652677424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:54:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:54:33.312896601 +0000 UTC m=+329.965087470" watchObservedRunningTime="2026-07-26 17:54:33.314872225 +0000 UTC m=+329.967063083"
Jul 26 17:54:34.550154 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:34.550092    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8"]
Jul 26 17:54:34.552804 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:34.552786    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8"
Jul 26 17:54:34.554855 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:34.554832    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 26 17:54:34.554965 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:34.554890    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-b628c\""
Jul 26 17:54:34.554965 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:34.554948    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 26 17:54:34.561953 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:34.561931    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8"]
Jul 26 17:54:34.642970 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:34.642927    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/baf5981a-18ec-4d01-94f5-3718d092000a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8\" (UID: \"baf5981a-18ec-4d01-94f5-3718d092000a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8"
Jul 26 17:54:34.642970 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:34.642971    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/baf5981a-18ec-4d01-94f5-3718d092000a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8\" (UID: \"baf5981a-18ec-4d01-94f5-3718d092000a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8"
Jul 26 17:54:34.643280 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:34.642997    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8m8c\" (UniqueName: \"kubernetes.io/projected/baf5981a-18ec-4d01-94f5-3718d092000a-kube-api-access-z8m8c\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8\" (UID: \"baf5981a-18ec-4d01-94f5-3718d092000a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8"
Jul 26 17:54:34.743621 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:34.743584    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/baf5981a-18ec-4d01-94f5-3718d092000a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8\" (UID: \"baf5981a-18ec-4d01-94f5-3718d092000a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8"
Jul 26 17:54:34.743795 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:34.743627    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/baf5981a-18ec-4d01-94f5-3718d092000a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8\" (UID: \"baf5981a-18ec-4d01-94f5-3718d092000a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8"
Jul 26 17:54:34.743795 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:34.743651    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8m8c\" (UniqueName: \"kubernetes.io/projected/baf5981a-18ec-4d01-94f5-3718d092000a-kube-api-access-z8m8c\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8\" (UID: \"baf5981a-18ec-4d01-94f5-3718d092000a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8"
Jul 26 17:54:34.743991 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:34.743973    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/baf5981a-18ec-4d01-94f5-3718d092000a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8\" (UID: \"baf5981a-18ec-4d01-94f5-3718d092000a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8"
Jul 26 17:54:34.744028 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:34.744000    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/baf5981a-18ec-4d01-94f5-3718d092000a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8\" (UID: \"baf5981a-18ec-4d01-94f5-3718d092000a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8"
Jul 26 17:54:34.752170 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:34.752143    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8m8c\" (UniqueName: \"kubernetes.io/projected/baf5981a-18ec-4d01-94f5-3718d092000a-kube-api-access-z8m8c\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8\" (UID: \"baf5981a-18ec-4d01-94f5-3718d092000a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8"
Jul 26 17:54:34.862357 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:34.862275    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8"
Jul 26 17:54:34.996588 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:34.996552    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8"]
Jul 26 17:54:34.999002 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:54:34.998975    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaf5981a_18ec_4d01_94f5_3718d092000a.slice/crio-1335a86d59808fcdc09f2308103265dce0356793f741cae01035e765d373e6d7 WatchSource:0}: Error finding container 1335a86d59808fcdc09f2308103265dce0356793f741cae01035e765d373e6d7: Status 404 returned error can't find the container with id 1335a86d59808fcdc09f2308103265dce0356793f741cae01035e765d373e6d7
Jul 26 17:54:35.298459 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:35.298427    2570 generic.go:356] "Generic (PLEG): container finished" podID="baf5981a-18ec-4d01-94f5-3718d092000a" containerID="9d07c98832b8b573cf36f4b45f05cf4d41c1a167dfad99833c7251b7b2a03ef1" exitCode=0
Jul 26 17:54:35.298649 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:35.298522    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8" event={"ID":"baf5981a-18ec-4d01-94f5-3718d092000a","Type":"ContainerDied","Data":"9d07c98832b8b573cf36f4b45f05cf4d41c1a167dfad99833c7251b7b2a03ef1"}
Jul 26 17:54:35.298649 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:35.298580    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8" event={"ID":"baf5981a-18ec-4d01-94f5-3718d092000a","Type":"ContainerStarted","Data":"1335a86d59808fcdc09f2308103265dce0356793f741cae01035e765d373e6d7"}
Jul 26 17:54:38.313095 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:38.313059    2570 generic.go:356] "Generic (PLEG): container finished" podID="baf5981a-18ec-4d01-94f5-3718d092000a" containerID="3d1bfbd578cd63366283a28aa1205e747f520b5dc805704b6cb877ddcbbf5614" exitCode=0
Jul 26 17:54:38.313517 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:38.313156    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8" event={"ID":"baf5981a-18ec-4d01-94f5-3718d092000a","Type":"ContainerDied","Data":"3d1bfbd578cd63366283a28aa1205e747f520b5dc805704b6cb877ddcbbf5614"}
Jul 26 17:54:39.318921 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:39.318884    2570 generic.go:356] "Generic (PLEG): container finished" podID="baf5981a-18ec-4d01-94f5-3718d092000a" containerID="9548a3ffaace6c31129ebf3b708056d52b14beda92a15592b44cca86566b299a" exitCode=0
Jul 26 17:54:39.319370 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:39.318957    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8" event={"ID":"baf5981a-18ec-4d01-94f5-3718d092000a","Type":"ContainerDied","Data":"9548a3ffaace6c31129ebf3b708056d52b14beda92a15592b44cca86566b299a"}
Jul 26 17:54:40.449070 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:40.449045    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8"
Jul 26 17:54:40.595991 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:40.595901    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/baf5981a-18ec-4d01-94f5-3718d092000a-bundle\") pod \"baf5981a-18ec-4d01-94f5-3718d092000a\" (UID: \"baf5981a-18ec-4d01-94f5-3718d092000a\") "
Jul 26 17:54:40.595991 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:40.595970    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8m8c\" (UniqueName: \"kubernetes.io/projected/baf5981a-18ec-4d01-94f5-3718d092000a-kube-api-access-z8m8c\") pod \"baf5981a-18ec-4d01-94f5-3718d092000a\" (UID: \"baf5981a-18ec-4d01-94f5-3718d092000a\") "
Jul 26 17:54:40.595991 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:40.595993    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/baf5981a-18ec-4d01-94f5-3718d092000a-util\") pod \"baf5981a-18ec-4d01-94f5-3718d092000a\" (UID: \"baf5981a-18ec-4d01-94f5-3718d092000a\") "
Jul 26 17:54:40.596381 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:40.596356    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baf5981a-18ec-4d01-94f5-3718d092000a-bundle" (OuterVolumeSpecName: "bundle") pod "baf5981a-18ec-4d01-94f5-3718d092000a" (UID: "baf5981a-18ec-4d01-94f5-3718d092000a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 17:54:40.598069 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:40.598041    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baf5981a-18ec-4d01-94f5-3718d092000a-kube-api-access-z8m8c" (OuterVolumeSpecName: "kube-api-access-z8m8c") pod "baf5981a-18ec-4d01-94f5-3718d092000a" (UID: "baf5981a-18ec-4d01-94f5-3718d092000a"). InnerVolumeSpecName "kube-api-access-z8m8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:54:40.600505 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:40.600482    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baf5981a-18ec-4d01-94f5-3718d092000a-util" (OuterVolumeSpecName: "util") pod "baf5981a-18ec-4d01-94f5-3718d092000a" (UID: "baf5981a-18ec-4d01-94f5-3718d092000a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 17:54:40.696759 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:40.696708    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8m8c\" (UniqueName: \"kubernetes.io/projected/baf5981a-18ec-4d01-94f5-3718d092000a-kube-api-access-z8m8c\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:54:40.696759 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:40.696752    2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/baf5981a-18ec-4d01-94f5-3718d092000a-util\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:54:40.696759 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:40.696767    2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/baf5981a-18ec-4d01-94f5-3718d092000a-bundle\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:54:41.328705 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:41.328665    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8" event={"ID":"baf5981a-18ec-4d01-94f5-3718d092000a","Type":"ContainerDied","Data":"1335a86d59808fcdc09f2308103265dce0356793f741cae01035e765d373e6d7"}
Jul 26 17:54:41.328879 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:41.328715    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhvxm8"
Jul 26 17:54:41.328879 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:41.328724    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1335a86d59808fcdc09f2308103265dce0356793f741cae01035e765d373e6d7"
Jul 26 17:54:42.053102 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:42.053074    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-z6h44"]
Jul 26 17:54:42.053455 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:42.053403    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baf5981a-18ec-4d01-94f5-3718d092000a" containerName="pull"
Jul 26 17:54:42.053455 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:42.053414    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="baf5981a-18ec-4d01-94f5-3718d092000a" containerName="pull"
Jul 26 17:54:42.053455 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:42.053423    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baf5981a-18ec-4d01-94f5-3718d092000a" containerName="extract"
Jul 26 17:54:42.053455 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:42.053428    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="baf5981a-18ec-4d01-94f5-3718d092000a" containerName="extract"
Jul 26 17:54:42.053455 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:42.053438    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baf5981a-18ec-4d01-94f5-3718d092000a" containerName="util"
Jul 26 17:54:42.053455 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:42.053443    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="baf5981a-18ec-4d01-94f5-3718d092000a" containerName="util"
Jul 26 17:54:42.053642 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:42.053491    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="baf5981a-18ec-4d01-94f5-3718d092000a" containerName="extract"
Jul 26 17:54:42.057762 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:42.057736    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-z6h44"
Jul 26 17:54:42.059872 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:42.059850    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 26 17:54:42.060528 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:42.060507    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-w4w7z\""
Jul 26 17:54:42.060641 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:42.060562    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 26 17:54:42.063940 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:42.063840    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-z6h44"]
Jul 26 17:54:42.209423 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:42.209326    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmv8n\" (UniqueName: \"kubernetes.io/projected/50f27ab6-ba67-426e-bb64-a93a7eb5d24d-kube-api-access-gmv8n\") pod \"cert-manager-cainjector-559cdd8c87-z6h44\" (UID: \"50f27ab6-ba67-426e-bb64-a93a7eb5d24d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-z6h44"
Jul 26 17:54:42.209423 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:42.209409    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50f27ab6-ba67-426e-bb64-a93a7eb5d24d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-z6h44\" (UID: \"50f27ab6-ba67-426e-bb64-a93a7eb5d24d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-z6h44"
Jul 26 17:54:42.310662 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:42.310627    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50f27ab6-ba67-426e-bb64-a93a7eb5d24d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-z6h44\" (UID: \"50f27ab6-ba67-426e-bb64-a93a7eb5d24d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-z6h44"
Jul 26 17:54:42.310842 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:42.310676    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmv8n\" (UniqueName: \"kubernetes.io/projected/50f27ab6-ba67-426e-bb64-a93a7eb5d24d-kube-api-access-gmv8n\") pod \"cert-manager-cainjector-559cdd8c87-z6h44\" (UID: \"50f27ab6-ba67-426e-bb64-a93a7eb5d24d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-z6h44"
Jul 26 17:54:42.319170 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:42.319138    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50f27ab6-ba67-426e-bb64-a93a7eb5d24d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-z6h44\" (UID: \"50f27ab6-ba67-426e-bb64-a93a7eb5d24d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-z6h44"
Jul 26 17:54:42.319338 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:42.319221    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmv8n\" (UniqueName: \"kubernetes.io/projected/50f27ab6-ba67-426e-bb64-a93a7eb5d24d-kube-api-access-gmv8n\") pod \"cert-manager-cainjector-559cdd8c87-z6h44\" (UID: \"50f27ab6-ba67-426e-bb64-a93a7eb5d24d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-z6h44"
Jul 26 17:54:42.384553 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:42.384516    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-z6h44"
Jul 26 17:54:42.511032 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:42.510961    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-z6h44"]
Jul 26 17:54:42.513649 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:54:42.513618    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50f27ab6_ba67_426e_bb64_a93a7eb5d24d.slice/crio-7f6ad15483a787d1c144482a116bc247458080a4752ec8002aaf8c54cdaafdd3 WatchSource:0}: Error finding container 7f6ad15483a787d1c144482a116bc247458080a4752ec8002aaf8c54cdaafdd3: Status 404 returned error can't find the container with id 7f6ad15483a787d1c144482a116bc247458080a4752ec8002aaf8c54cdaafdd3
Jul 26 17:54:43.344820 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:43.344780    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-z6h44" event={"ID":"50f27ab6-ba67-426e-bb64-a93a7eb5d24d","Type":"ContainerStarted","Data":"7f6ad15483a787d1c144482a116bc247458080a4752ec8002aaf8c54cdaafdd3"}
Jul 26 17:54:45.356574 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:45.356533    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-z6h44" event={"ID":"50f27ab6-ba67-426e-bb64-a93a7eb5d24d","Type":"ContainerStarted","Data":"2d3048cf7b2822cd24e0702db73eb2613d76824a7e39316266eaed5fde9068eb"}
Jul 26 17:54:45.373875 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:45.373827    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-z6h44" podStartSLOduration=0.887737053 podStartE2EDuration="3.373810998s" totalImagesPullingTime="2.486073945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:54:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:54:45.372341992 +0000 UTC m=+342.024532850" watchObservedRunningTime="2026-07-26 17:54:45.373810998 +0000 UTC m=+342.026001855"
Jul 26 17:54:55.425417 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:55.425374    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw"]
Jul 26 17:54:55.428866 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:55.428848    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw"
Jul 26 17:54:55.430946 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:55.430924    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 26 17:54:55.431494 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:55.431479    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 26 17:54:55.431494 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:55.431485    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-b628c\""
Jul 26 17:54:55.436520 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:55.436498    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw"]
Jul 26 17:54:55.506533 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:55.506502    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42364d09-e853-4903-9900-5234f508a169-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw\" (UID: \"42364d09-e853-4903-9900-5234f508a169\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw"
Jul 26 17:54:55.506533 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:55.506538    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42364d09-e853-4903-9900-5234f508a169-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw\" (UID: \"42364d09-e853-4903-9900-5234f508a169\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw"
Jul 26 17:54:55.506739 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:55.506559    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rclz5\" (UniqueName: \"kubernetes.io/projected/42364d09-e853-4903-9900-5234f508a169-kube-api-access-rclz5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw\" (UID: \"42364d09-e853-4903-9900-5234f508a169\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw"
Jul 26 17:54:55.606941 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:55.606904    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42364d09-e853-4903-9900-5234f508a169-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw\" (UID: \"42364d09-e853-4903-9900-5234f508a169\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw"
Jul 26 17:54:55.606941 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:55.606944    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42364d09-e853-4903-9900-5234f508a169-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw\" (UID: \"42364d09-e853-4903-9900-5234f508a169\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw"
Jul 26 17:54:55.607177 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:55.606968    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rclz5\" (UniqueName: \"kubernetes.io/projected/42364d09-e853-4903-9900-5234f508a169-kube-api-access-rclz5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw\" (UID: \"42364d09-e853-4903-9900-5234f508a169\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw"
Jul 26 17:54:55.607333 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:55.607314    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42364d09-e853-4903-9900-5234f508a169-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw\" (UID: \"42364d09-e853-4903-9900-5234f508a169\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw"
Jul 26 17:54:55.607383 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:55.607363    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42364d09-e853-4903-9900-5234f508a169-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw\" (UID: \"42364d09-e853-4903-9900-5234f508a169\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw"
Jul 26 17:54:55.614761 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:55.614737    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rclz5\" (UniqueName: \"kubernetes.io/projected/42364d09-e853-4903-9900-5234f508a169-kube-api-access-rclz5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw\" (UID: \"42364d09-e853-4903-9900-5234f508a169\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw"
Jul 26 17:54:55.738059 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:55.737974    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw"
Jul 26 17:54:55.890000 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:55.889968    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw"]
Jul 26 17:54:55.893530 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:54:55.893496    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42364d09_e853_4903_9900_5234f508a169.slice/crio-cd74c7c8d5c5a97af9e38f5252c7e486d3e0028a06798ebfe61473af7cbf058c WatchSource:0}: Error finding container cd74c7c8d5c5a97af9e38f5252c7e486d3e0028a06798ebfe61473af7cbf058c: Status 404 returned error can't find the container with id cd74c7c8d5c5a97af9e38f5252c7e486d3e0028a06798ebfe61473af7cbf058c
Jul 26 17:54:56.408586 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:56.408475    2570 generic.go:356] "Generic (PLEG): container finished" podID="42364d09-e853-4903-9900-5234f508a169" containerID="5052de6b3f6dc14001f8e9573db1cef12103d0f3566cba7c8cfe1eaf67fd0d54" exitCode=0
Jul 26 17:54:56.408586 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:56.408556    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw" event={"ID":"42364d09-e853-4903-9900-5234f508a169","Type":"ContainerDied","Data":"5052de6b3f6dc14001f8e9573db1cef12103d0f3566cba7c8cfe1eaf67fd0d54"}
Jul 26 17:54:56.408796 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:56.408616    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw" event={"ID":"42364d09-e853-4903-9900-5234f508a169","Type":"ContainerStarted","Data":"cd74c7c8d5c5a97af9e38f5252c7e486d3e0028a06798ebfe61473af7cbf058c"}
Jul 26 17:54:58.418482 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:58.418449    2570 generic.go:356] "Generic (PLEG): container finished" podID="42364d09-e853-4903-9900-5234f508a169" containerID="1db796160d4dffa53f47b1bcd712741b69a38db59c78412c225573daf5e24fdd" exitCode=0
Jul 26 17:54:58.418864 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:58.418530    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw" event={"ID":"42364d09-e853-4903-9900-5234f508a169","Type":"ContainerDied","Data":"1db796160d4dffa53f47b1bcd712741b69a38db59c78412c225573daf5e24fdd"}
Jul 26 17:54:59.425781 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:59.425750    2570 generic.go:356] "Generic (PLEG): container finished" podID="42364d09-e853-4903-9900-5234f508a169" containerID="674e83c0ad299a4f2736da0479db42910c8216a577e8390eff0aebae528ad506" exitCode=0
Jul 26 17:54:59.426184 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:54:59.425827    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw" event={"ID":"42364d09-e853-4903-9900-5234f508a169","Type":"ContainerDied","Data":"674e83c0ad299a4f2736da0479db42910c8216a577e8390eff0aebae528ad506"}
Jul 26 17:55:00.567771 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:00.567740    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw"
Jul 26 17:55:00.641555 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:00.641523    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42364d09-e853-4903-9900-5234f508a169-util\") pod \"42364d09-e853-4903-9900-5234f508a169\" (UID: \"42364d09-e853-4903-9900-5234f508a169\") "
Jul 26 17:55:00.641717 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:00.641572    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rclz5\" (UniqueName: \"kubernetes.io/projected/42364d09-e853-4903-9900-5234f508a169-kube-api-access-rclz5\") pod \"42364d09-e853-4903-9900-5234f508a169\" (UID: \"42364d09-e853-4903-9900-5234f508a169\") "
Jul 26 17:55:00.641717 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:00.641600    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42364d09-e853-4903-9900-5234f508a169-bundle\") pod \"42364d09-e853-4903-9900-5234f508a169\" (UID: \"42364d09-e853-4903-9900-5234f508a169\") "
Jul 26 17:55:00.642399 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:00.642373    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42364d09-e853-4903-9900-5234f508a169-bundle" (OuterVolumeSpecName: "bundle") pod "42364d09-e853-4903-9900-5234f508a169" (UID: "42364d09-e853-4903-9900-5234f508a169"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 17:55:00.643694 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:00.643671    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42364d09-e853-4903-9900-5234f508a169-kube-api-access-rclz5" (OuterVolumeSpecName: "kube-api-access-rclz5") pod "42364d09-e853-4903-9900-5234f508a169" (UID: "42364d09-e853-4903-9900-5234f508a169"). InnerVolumeSpecName "kube-api-access-rclz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:55:00.649840 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:00.649812    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42364d09-e853-4903-9900-5234f508a169-util" (OuterVolumeSpecName: "util") pod "42364d09-e853-4903-9900-5234f508a169" (UID: "42364d09-e853-4903-9900-5234f508a169"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 17:55:00.742469 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:00.742386    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rclz5\" (UniqueName: \"kubernetes.io/projected/42364d09-e853-4903-9900-5234f508a169-kube-api-access-rclz5\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:55:00.742469 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:00.742417    2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42364d09-e853-4903-9900-5234f508a169-bundle\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:55:00.742469 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:00.742427    2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42364d09-e853-4903-9900-5234f508a169-util\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:55:01.440546 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:01.440516    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw"
Jul 26 17:55:01.440546 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:01.440538    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8wwmw" event={"ID":"42364d09-e853-4903-9900-5234f508a169","Type":"ContainerDied","Data":"cd74c7c8d5c5a97af9e38f5252c7e486d3e0028a06798ebfe61473af7cbf058c"}
Jul 26 17:55:01.440754 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:01.440577    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd74c7c8d5c5a97af9e38f5252c7e486d3e0028a06798ebfe61473af7cbf058c"
Jul 26 17:55:08.105220 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.105184    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6"]
Jul 26 17:55:08.105607 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.105466    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42364d09-e853-4903-9900-5234f508a169" containerName="extract"
Jul 26 17:55:08.105607 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.105477    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="42364d09-e853-4903-9900-5234f508a169" containerName="extract"
Jul 26 17:55:08.105607 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.105487    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42364d09-e853-4903-9900-5234f508a169" containerName="pull"
Jul 26 17:55:08.105607 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.105493    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="42364d09-e853-4903-9900-5234f508a169" containerName="pull"
Jul 26 17:55:08.105607 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.105514    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42364d09-e853-4903-9900-5234f508a169" containerName="util"
Jul 26 17:55:08.105607 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.105519    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="42364d09-e853-4903-9900-5234f508a169" containerName="util"
Jul 26 17:55:08.105607 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.105567    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="42364d09-e853-4903-9900-5234f508a169" containerName="extract"
Jul 26 17:55:08.111294 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.111272    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6"
Jul 26 17:55:08.115098 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.115077    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 26 17:55:08.115319 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.115305    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 26 17:55:08.115421 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.115402    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ml4x7\""
Jul 26 17:55:08.120504 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.120484    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 26 17:55:08.120600 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.120512    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 26 17:55:08.136225 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.136192    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6"]
Jul 26 17:55:08.202443 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.202407    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh6c5\" (UniqueName: \"kubernetes.io/projected/86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa-kube-api-access-kh6c5\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-l2rp6\" (UID: \"86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6"
Jul 26 17:55:08.202443 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.202449    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa-tmp\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-l2rp6\" (UID: \"86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6"
Jul 26 17:55:08.202657 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.202468    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-l2rp6\" (UID: \"86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6"
Jul 26 17:55:08.202657 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.202503    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-l2rp6\" (UID: \"86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6"
Jul 26 17:55:08.202657 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.202541    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa-manifests\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-l2rp6\" (UID: \"86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6"
Jul 26 17:55:08.303130 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.303089    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh6c5\" (UniqueName: \"kubernetes.io/projected/86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa-kube-api-access-kh6c5\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-l2rp6\" (UID: \"86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6"
Jul 26 17:55:08.303309 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.303153    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa-tmp\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-l2rp6\" (UID: \"86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6"
Jul 26 17:55:08.303309 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.303177    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-l2rp6\" (UID: \"86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6"
Jul 26 17:55:08.303309 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.303202    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-l2rp6\" (UID: \"86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6"
Jul 26 17:55:08.303423 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.303354    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa-manifests\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-l2rp6\" (UID: \"86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6"
Jul 26 17:55:08.303701 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.303675    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa-manifests\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-l2rp6\" (UID: \"86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6"
Jul 26 17:55:08.305626 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.305602    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa-tmp\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-l2rp6\" (UID: \"86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6"
Jul 26 17:55:08.305759 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.305743    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-l2rp6\" (UID: \"86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6"
Jul 26 17:55:08.305931 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.305911    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-l2rp6\" (UID: \"86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6"
Jul 26 17:55:08.310971 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.310947    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh6c5\" (UniqueName: \"kubernetes.io/projected/86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa-kube-api-access-kh6c5\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-l2rp6\" (UID: \"86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6"
Jul 26 17:55:08.422103 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.422002    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6"
Jul 26 17:55:08.558244 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:08.558217    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6"]
Jul 26 17:55:08.560324 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:55:08.560296    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86e0625a_e1d7_4cd8_9f6d_b8c0d50c50fa.slice/crio-b83c7676a90357d73c1c7c19043becc30de7fe3d9611b4594d09ea6e3bcc41d8 WatchSource:0}: Error finding container b83c7676a90357d73c1c7c19043becc30de7fe3d9611b4594d09ea6e3bcc41d8: Status 404 returned error can't find the container with id b83c7676a90357d73c1c7c19043becc30de7fe3d9611b4594d09ea6e3bcc41d8
Jul 26 17:55:09.483309 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:09.483229    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6" event={"ID":"86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa","Type":"ContainerStarted","Data":"b83c7676a90357d73c1c7c19043becc30de7fe3d9611b4594d09ea6e3bcc41d8"}
Jul 26 17:55:11.494211 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:11.494179    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6" event={"ID":"86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa","Type":"ContainerStarted","Data":"57c004b0c387e3bf5ec4e314e731affdcbc02118064db023942fa78b68440c15"}
Jul 26 17:55:13.504659 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:13.504622    2570 generic.go:356] "Generic (PLEG): container finished" podID="86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa" containerID="57c004b0c387e3bf5ec4e314e731affdcbc02118064db023942fa78b68440c15" exitCode=0
Jul 26 17:55:13.505082 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:13.504685    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6" event={"ID":"86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa","Type":"ContainerDied","Data":"57c004b0c387e3bf5ec4e314e731affdcbc02118064db023942fa78b68440c15"}
Jul 26 17:55:14.511211 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:14.511180    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6" event={"ID":"86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa","Type":"ContainerStarted","Data":"30bf7d56dcf075f1df47be2087dae7ba6a6ed634be1214c7b2eaacfea2cf1afd"}
Jul 26 17:55:14.511654 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:14.511404    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6"
Jul 26 17:55:14.539373 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:14.539316    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6" podStartSLOduration=2.326602369 podStartE2EDuration="6.539297253s" totalImagesPullingTime="3.143836994s" totalInitContainerRuntime="1.06885789s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:55:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:55:14.53833508 +0000 UTC m=+371.190525948" watchObservedRunningTime="2026-07-26 17:55:14.539297253 +0000 UTC m=+371.191488116"
Jul 26 17:55:25.517623 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:25.517590    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-l2rp6"
Jul 26 17:55:28.978876 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:28.978839    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6855b89bd5-rrbbr"]
Jul 26 17:55:28.982093 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:28.982076    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6855b89bd5-rrbbr"
Jul 26 17:55:28.985429 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:28.985408    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-r79c6\""
Jul 26 17:55:28.985987 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:28.985970    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 26 17:55:28.986058 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:28.985970    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 26 17:55:29.003886 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.003853    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6855b89bd5-rrbbr"]
Jul 26 17:55:29.075009 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.074976    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4v78\" (UniqueName: \"kubernetes.io/projected/10d138c1-d703-4e3b-b6e9-261b93300cfe-kube-api-access-p4v78\") pod \"kube-auth-proxy-6855b89bd5-rrbbr\" (UID: \"10d138c1-d703-4e3b-b6e9-261b93300cfe\") " pod="openshift-ingress/kube-auth-proxy-6855b89bd5-rrbbr"
Jul 26 17:55:29.075169 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.075035    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10d138c1-d703-4e3b-b6e9-261b93300cfe-tls-certs\") pod \"kube-auth-proxy-6855b89bd5-rrbbr\" (UID: \"10d138c1-d703-4e3b-b6e9-261b93300cfe\") " pod="openshift-ingress/kube-auth-proxy-6855b89bd5-rrbbr"
Jul 26 17:55:29.075169 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.075060    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10d138c1-d703-4e3b-b6e9-261b93300cfe-tmp\") pod \"kube-auth-proxy-6855b89bd5-rrbbr\" (UID: \"10d138c1-d703-4e3b-b6e9-261b93300cfe\") " pod="openshift-ingress/kube-auth-proxy-6855b89bd5-rrbbr"
Jul 26 17:55:29.176140 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.176078    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10d138c1-d703-4e3b-b6e9-261b93300cfe-tls-certs\") pod \"kube-auth-proxy-6855b89bd5-rrbbr\" (UID: \"10d138c1-d703-4e3b-b6e9-261b93300cfe\") " pod="openshift-ingress/kube-auth-proxy-6855b89bd5-rrbbr"
Jul 26 17:55:29.176140 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.176144    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10d138c1-d703-4e3b-b6e9-261b93300cfe-tmp\") pod \"kube-auth-proxy-6855b89bd5-rrbbr\" (UID: \"10d138c1-d703-4e3b-b6e9-261b93300cfe\") " pod="openshift-ingress/kube-auth-proxy-6855b89bd5-rrbbr"
Jul 26 17:55:29.176368 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.176188    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4v78\" (UniqueName: \"kubernetes.io/projected/10d138c1-d703-4e3b-b6e9-261b93300cfe-kube-api-access-p4v78\") pod \"kube-auth-proxy-6855b89bd5-rrbbr\" (UID: \"10d138c1-d703-4e3b-b6e9-261b93300cfe\") " pod="openshift-ingress/kube-auth-proxy-6855b89bd5-rrbbr"
Jul 26 17:55:29.178398 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.178364    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10d138c1-d703-4e3b-b6e9-261b93300cfe-tmp\") pod \"kube-auth-proxy-6855b89bd5-rrbbr\" (UID: \"10d138c1-d703-4e3b-b6e9-261b93300cfe\") " pod="openshift-ingress/kube-auth-proxy-6855b89bd5-rrbbr"
Jul 26 17:55:29.178606 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.178587    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10d138c1-d703-4e3b-b6e9-261b93300cfe-tls-certs\") pod \"kube-auth-proxy-6855b89bd5-rrbbr\" (UID: \"10d138c1-d703-4e3b-b6e9-261b93300cfe\") " pod="openshift-ingress/kube-auth-proxy-6855b89bd5-rrbbr"
Jul 26 17:55:29.193586 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.193555    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4v78\" (UniqueName: \"kubernetes.io/projected/10d138c1-d703-4e3b-b6e9-261b93300cfe-kube-api-access-p4v78\") pod \"kube-auth-proxy-6855b89bd5-rrbbr\" (UID: \"10d138c1-d703-4e3b-b6e9-261b93300cfe\") " pod="openshift-ingress/kube-auth-proxy-6855b89bd5-rrbbr"
Jul 26 17:55:29.241869 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.241793    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jfjsd"]
Jul 26 17:55:29.245076 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.245054    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-jfjsd"
Jul 26 17:55:29.247806 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.247787    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-7x2nr\""
Jul 26 17:55:29.248166 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.248149    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 26 17:55:29.255246 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.255224    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jfjsd"]
Jul 26 17:55:29.291478 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.291442    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6855b89bd5-rrbbr"
Jul 26 17:55:29.378786 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.378753    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d82hg\" (UniqueName: \"kubernetes.io/projected/318aa469-e0e0-4985-8f40-01b5557ca6eb-kube-api-access-d82hg\") pod \"odh-model-controller-7998c67dc4-jfjsd\" (UID: \"318aa469-e0e0-4985-8f40-01b5557ca6eb\") " pod="opendatahub/odh-model-controller-7998c67dc4-jfjsd"
Jul 26 17:55:29.378940 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.378822    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/318aa469-e0e0-4985-8f40-01b5557ca6eb-cert\") pod \"odh-model-controller-7998c67dc4-jfjsd\" (UID: \"318aa469-e0e0-4985-8f40-01b5557ca6eb\") " pod="opendatahub/odh-model-controller-7998c67dc4-jfjsd"
Jul 26 17:55:29.426032 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.426008    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6855b89bd5-rrbbr"]
Jul 26 17:55:29.427664 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:55:29.427637    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10d138c1_d703_4e3b_b6e9_261b93300cfe.slice/crio-015f1efa43483a9a16eafa427f25fb65f6ae58d05b013525590f921e3d3c5714 WatchSource:0}: Error finding container 015f1efa43483a9a16eafa427f25fb65f6ae58d05b013525590f921e3d3c5714: Status 404 returned error can't find the container with id 015f1efa43483a9a16eafa427f25fb65f6ae58d05b013525590f921e3d3c5714
Jul 26 17:55:29.480204 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.480159    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d82hg\" (UniqueName: \"kubernetes.io/projected/318aa469-e0e0-4985-8f40-01b5557ca6eb-kube-api-access-d82hg\") pod \"odh-model-controller-7998c67dc4-jfjsd\" (UID: \"318aa469-e0e0-4985-8f40-01b5557ca6eb\") " pod="opendatahub/odh-model-controller-7998c67dc4-jfjsd"
Jul 26 17:55:29.480373 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.480327    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/318aa469-e0e0-4985-8f40-01b5557ca6eb-cert\") pod \"odh-model-controller-7998c67dc4-jfjsd\" (UID: \"318aa469-e0e0-4985-8f40-01b5557ca6eb\") " pod="opendatahub/odh-model-controller-7998c67dc4-jfjsd"
Jul 26 17:55:29.480483 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:55:29.480464    2570 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 26 17:55:29.480541 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:55:29.480531    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/318aa469-e0e0-4985-8f40-01b5557ca6eb-cert podName:318aa469-e0e0-4985-8f40-01b5557ca6eb nodeName:}" failed. No retries permitted until 2026-07-26 17:55:29.980516477 +0000 UTC m=+386.632707313 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/318aa469-e0e0-4985-8f40-01b5557ca6eb-cert") pod "odh-model-controller-7998c67dc4-jfjsd" (UID: "318aa469-e0e0-4985-8f40-01b5557ca6eb") : secret "odh-model-controller-webhook-cert" not found
Jul 26 17:55:29.491096 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.491070    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d82hg\" (UniqueName: \"kubernetes.io/projected/318aa469-e0e0-4985-8f40-01b5557ca6eb-kube-api-access-d82hg\") pod \"odh-model-controller-7998c67dc4-jfjsd\" (UID: \"318aa469-e0e0-4985-8f40-01b5557ca6eb\") " pod="opendatahub/odh-model-controller-7998c67dc4-jfjsd"
Jul 26 17:55:29.577693 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.577654    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6855b89bd5-rrbbr" event={"ID":"10d138c1-d703-4e3b-b6e9-261b93300cfe","Type":"ContainerStarted","Data":"015f1efa43483a9a16eafa427f25fb65f6ae58d05b013525590f921e3d3c5714"}
Jul 26 17:55:29.984558 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.984462    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/318aa469-e0e0-4985-8f40-01b5557ca6eb-cert\") pod \"odh-model-controller-7998c67dc4-jfjsd\" (UID: \"318aa469-e0e0-4985-8f40-01b5557ca6eb\") " pod="opendatahub/odh-model-controller-7998c67dc4-jfjsd"
Jul 26 17:55:29.986825 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:29.986804    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/318aa469-e0e0-4985-8f40-01b5557ca6eb-cert\") pod \"odh-model-controller-7998c67dc4-jfjsd\" (UID: \"318aa469-e0e0-4985-8f40-01b5557ca6eb\") " pod="opendatahub/odh-model-controller-7998c67dc4-jfjsd"
Jul 26 17:55:30.156353 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.156315    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-jfjsd"
Jul 26 17:55:30.223504 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.223452    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"]
Jul 26 17:55:30.230365 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.230340    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:30.233310 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.233169    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 26 17:55:30.233310 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.233176    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 26 17:55:30.233494 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.233368    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-pg7gm\""
Jul 26 17:55:30.233607 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.233590    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 26 17:55:30.233932 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.233590    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 26 17:55:30.242159 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.242029    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"]
Jul 26 17:55:30.287820 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.287789    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/118b15c4-0aa6-40b0-b550-2836c38d07c7-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-7sfr4\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:30.287995 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.287836    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/118b15c4-0aa6-40b0-b550-2836c38d07c7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-7sfr4\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:30.287995 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.287886    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/118b15c4-0aa6-40b0-b550-2836c38d07c7-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-7sfr4\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:30.287995 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.287928    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqrwh\" (UniqueName: \"kubernetes.io/projected/118b15c4-0aa6-40b0-b550-2836c38d07c7-kube-api-access-tqrwh\") pod \"istiod-openshift-gateway-7b7995d5cd-7sfr4\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:30.287995 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.287967    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/118b15c4-0aa6-40b0-b550-2836c38d07c7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-7sfr4\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:30.288221 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.287995    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/118b15c4-0aa6-40b0-b550-2836c38d07c7-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-7sfr4\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:30.288221 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.288022    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/118b15c4-0aa6-40b0-b550-2836c38d07c7-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-7sfr4\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:30.323556 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.323527    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jfjsd"]
Jul 26 17:55:30.329354 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:55:30.329318    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod318aa469_e0e0_4985_8f40_01b5557ca6eb.slice/crio-8b4df8fd9817332def6ed2bd63cf274afe02f97b2c9e7074a90a1fb7a4adc277 WatchSource:0}: Error finding container 8b4df8fd9817332def6ed2bd63cf274afe02f97b2c9e7074a90a1fb7a4adc277: Status 404 returned error can't find the container with id 8b4df8fd9817332def6ed2bd63cf274afe02f97b2c9e7074a90a1fb7a4adc277
Jul 26 17:55:30.388849 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.388815    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/118b15c4-0aa6-40b0-b550-2836c38d07c7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-7sfr4\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:30.389044 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.388853    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/118b15c4-0aa6-40b0-b550-2836c38d07c7-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-7sfr4\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:30.389044 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.388874    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/118b15c4-0aa6-40b0-b550-2836c38d07c7-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-7sfr4\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:30.389044 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.388915    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/118b15c4-0aa6-40b0-b550-2836c38d07c7-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-7sfr4\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:30.389044 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.388943    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/118b15c4-0aa6-40b0-b550-2836c38d07c7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-7sfr4\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:30.389044 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.388988    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/118b15c4-0aa6-40b0-b550-2836c38d07c7-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-7sfr4\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:30.389044 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.389025    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqrwh\" (UniqueName: \"kubernetes.io/projected/118b15c4-0aa6-40b0-b550-2836c38d07c7-kube-api-access-tqrwh\") pod \"istiod-openshift-gateway-7b7995d5cd-7sfr4\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:30.389894 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.389869    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/118b15c4-0aa6-40b0-b550-2836c38d07c7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-7sfr4\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:30.391327 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.391307    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/118b15c4-0aa6-40b0-b550-2836c38d07c7-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-7sfr4\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:30.391477 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.391450    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/118b15c4-0aa6-40b0-b550-2836c38d07c7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-7sfr4\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:30.391705 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.391686    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/118b15c4-0aa6-40b0-b550-2836c38d07c7-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-7sfr4\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:30.391814 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.391791    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/118b15c4-0aa6-40b0-b550-2836c38d07c7-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-7sfr4\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:30.397192 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.397092    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/118b15c4-0aa6-40b0-b550-2836c38d07c7-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-7sfr4\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:30.397436 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.397416    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqrwh\" (UniqueName: \"kubernetes.io/projected/118b15c4-0aa6-40b0-b550-2836c38d07c7-kube-api-access-tqrwh\") pod \"istiod-openshift-gateway-7b7995d5cd-7sfr4\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:30.544456 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.544418    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:30.585386 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.585345    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-jfjsd" event={"ID":"318aa469-e0e0-4985-8f40-01b5557ca6eb","Type":"ContainerStarted","Data":"8b4df8fd9817332def6ed2bd63cf274afe02f97b2c9e7074a90a1fb7a4adc277"}
Jul 26 17:55:30.703474 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:30.703449    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"]
Jul 26 17:55:30.705403 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:55:30.705379    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod118b15c4_0aa6_40b0_b550_2836c38d07c7.slice/crio-5a4295ca3f9804d428e1c61c6a75a093e926eba3fb5a6d27e97d778768fa7e52 WatchSource:0}: Error finding container 5a4295ca3f9804d428e1c61c6a75a093e926eba3fb5a6d27e97d778768fa7e52: Status 404 returned error can't find the container with id 5a4295ca3f9804d428e1c61c6a75a093e926eba3fb5a6d27e97d778768fa7e52
Jul 26 17:55:31.594102 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:31.594048    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4" event={"ID":"118b15c4-0aa6-40b0-b550-2836c38d07c7","Type":"ContainerStarted","Data":"5a4295ca3f9804d428e1c61c6a75a093e926eba3fb5a6d27e97d778768fa7e52"}
Jul 26 17:55:34.668057 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:34.668014    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 17:55:34.668476 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:34.668099    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 17:55:35.615707 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:35.615672    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-jfjsd" event={"ID":"318aa469-e0e0-4985-8f40-01b5557ca6eb","Type":"ContainerStarted","Data":"071095a6a659597f19ae43fde2887874dd601b3280f846fe8bf52411a2b456d1"}
Jul 26 17:55:35.615991 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:35.615795    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-jfjsd"
Jul 26 17:55:35.617650 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:35.617609    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6855b89bd5-rrbbr" event={"ID":"10d138c1-d703-4e3b-b6e9-261b93300cfe","Type":"ContainerStarted","Data":"442a4dbc07479eab9f0d6f05999a908e067e18a5af57184e4c86833886335b05"}
Jul 26 17:55:35.619403 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:35.619379    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4" event={"ID":"118b15c4-0aa6-40b0-b550-2836c38d07c7","Type":"ContainerStarted","Data":"7209ce1901ae6cf69446dccecf246f1c1b6585f09a902242fc31207e2748059f"}
Jul 26 17:55:35.619596 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:35.619579    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:35.621362 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:35.621338    2570 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-7sfr4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 26 17:55:35.621453 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:35.621388    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4" podUID="118b15c4-0aa6-40b0-b550-2836c38d07c7" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 26 17:55:35.633570 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:35.633530    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-jfjsd" podStartSLOduration=2.349935971 podStartE2EDuration="6.633516495s" totalImagesPullingTime="4.283580524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:55:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:55:35.631820609 +0000 UTC m=+392.284011480" watchObservedRunningTime="2026-07-26 17:55:35.633516495 +0000 UTC m=+392.285707387"
Jul 26 17:55:35.652532 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:35.652485    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6855b89bd5-rrbbr" podStartSLOduration=2.477671854 podStartE2EDuration="7.652469632s" totalImagesPullingTime="5.174797778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:55:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:55:35.651500365 +0000 UTC m=+392.303691224" watchObservedRunningTime="2026-07-26 17:55:35.652469632 +0000 UTC m=+392.304660555"
Jul 26 17:55:35.671832 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:35.671610    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4" podStartSLOduration=1.7113018530000002 podStartE2EDuration="5.671588373s" totalImagesPullingTime="3.96028652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:55:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:55:35.670901496 +0000 UTC m=+392.323092356" watchObservedRunningTime="2026-07-26 17:55:35.671588373 +0000 UTC m=+392.323779233"
Jul 26 17:55:36.625773 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:36.625746    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:39.178078 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:39.178030    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-fdddb58cf-z4cng"]
Jul 26 17:55:39.181570 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:39.181553    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-fdddb58cf-z4cng"
Jul 26 17:55:39.184595 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:39.184572    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 26 17:55:39.184595 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:39.184586    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 26 17:55:39.184797 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:39.184591    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 26 17:55:39.184851 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:39.184824    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 26 17:55:39.184898 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:39.184825    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-6tdb6\""
Jul 26 17:55:39.185072 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:39.185057    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 26 17:55:39.194589 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:39.194566    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-fdddb58cf-z4cng"]
Jul 26 17:55:39.276205 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:39.276165    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwhlj\" (UniqueName: \"kubernetes.io/projected/d382754b-8e98-4863-91ec-129ea9a09ac0-kube-api-access-pwhlj\") pod \"lws-controller-manager-fdddb58cf-z4cng\" (UID: \"d382754b-8e98-4863-91ec-129ea9a09ac0\") " pod="openshift-lws-operator/lws-controller-manager-fdddb58cf-z4cng"
Jul 26 17:55:39.276407 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:39.276221    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d382754b-8e98-4863-91ec-129ea9a09ac0-manager-config\") pod \"lws-controller-manager-fdddb58cf-z4cng\" (UID: \"d382754b-8e98-4863-91ec-129ea9a09ac0\") " pod="openshift-lws-operator/lws-controller-manager-fdddb58cf-z4cng"
Jul 26 17:55:39.276407 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:39.276247    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d382754b-8e98-4863-91ec-129ea9a09ac0-metrics-cert\") pod \"lws-controller-manager-fdddb58cf-z4cng\" (UID: \"d382754b-8e98-4863-91ec-129ea9a09ac0\") " pod="openshift-lws-operator/lws-controller-manager-fdddb58cf-z4cng"
Jul 26 17:55:39.276407 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:39.276305    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d382754b-8e98-4863-91ec-129ea9a09ac0-cert\") pod \"lws-controller-manager-fdddb58cf-z4cng\" (UID: \"d382754b-8e98-4863-91ec-129ea9a09ac0\") " pod="openshift-lws-operator/lws-controller-manager-fdddb58cf-z4cng"
Jul 26 17:55:39.377550 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:39.377502    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d382754b-8e98-4863-91ec-129ea9a09ac0-manager-config\") pod \"lws-controller-manager-fdddb58cf-z4cng\" (UID: \"d382754b-8e98-4863-91ec-129ea9a09ac0\") " pod="openshift-lws-operator/lws-controller-manager-fdddb58cf-z4cng"
Jul 26 17:55:39.377550 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:39.377553    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d382754b-8e98-4863-91ec-129ea9a09ac0-metrics-cert\") pod \"lws-controller-manager-fdddb58cf-z4cng\" (UID: \"d382754b-8e98-4863-91ec-129ea9a09ac0\") " pod="openshift-lws-operator/lws-controller-manager-fdddb58cf-z4cng"
Jul 26 17:55:39.377819 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:39.377581    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d382754b-8e98-4863-91ec-129ea9a09ac0-cert\") pod \"lws-controller-manager-fdddb58cf-z4cng\" (UID: \"d382754b-8e98-4863-91ec-129ea9a09ac0\") " pod="openshift-lws-operator/lws-controller-manager-fdddb58cf-z4cng"
Jul 26 17:55:39.377819 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:39.377656    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwhlj\" (UniqueName: \"kubernetes.io/projected/d382754b-8e98-4863-91ec-129ea9a09ac0-kube-api-access-pwhlj\") pod \"lws-controller-manager-fdddb58cf-z4cng\" (UID: \"d382754b-8e98-4863-91ec-129ea9a09ac0\") " pod="openshift-lws-operator/lws-controller-manager-fdddb58cf-z4cng"
Jul 26 17:55:39.378274 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:39.378251    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d382754b-8e98-4863-91ec-129ea9a09ac0-manager-config\") pod \"lws-controller-manager-fdddb58cf-z4cng\" (UID: \"d382754b-8e98-4863-91ec-129ea9a09ac0\") " pod="openshift-lws-operator/lws-controller-manager-fdddb58cf-z4cng"
Jul 26 17:55:39.380204 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:39.380173    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d382754b-8e98-4863-91ec-129ea9a09ac0-cert\") pod \"lws-controller-manager-fdddb58cf-z4cng\" (UID: \"d382754b-8e98-4863-91ec-129ea9a09ac0\") " pod="openshift-lws-operator/lws-controller-manager-fdddb58cf-z4cng"
Jul 26 17:55:39.380294 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:39.380251    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d382754b-8e98-4863-91ec-129ea9a09ac0-metrics-cert\") pod \"lws-controller-manager-fdddb58cf-z4cng\" (UID: \"d382754b-8e98-4863-91ec-129ea9a09ac0\") " pod="openshift-lws-operator/lws-controller-manager-fdddb58cf-z4cng"
Jul 26 17:55:39.395128 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:39.395084    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwhlj\" (UniqueName: \"kubernetes.io/projected/d382754b-8e98-4863-91ec-129ea9a09ac0-kube-api-access-pwhlj\") pod \"lws-controller-manager-fdddb58cf-z4cng\" (UID: \"d382754b-8e98-4863-91ec-129ea9a09ac0\") " pod="openshift-lws-operator/lws-controller-manager-fdddb58cf-z4cng"
Jul 26 17:55:39.491759 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:39.491681    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-fdddb58cf-z4cng"
Jul 26 17:55:39.633080 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:39.633053    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-fdddb58cf-z4cng"]
Jul 26 17:55:39.634663 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:55:39.634636    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd382754b_8e98_4863_91ec_129ea9a09ac0.slice/crio-46c4dcd7f3e34b80284966a050442f60656ffad88ce7303a899442b8944e416e WatchSource:0}: Error finding container 46c4dcd7f3e34b80284966a050442f60656ffad88ce7303a899442b8944e416e: Status 404 returned error can't find the container with id 46c4dcd7f3e34b80284966a050442f60656ffad88ce7303a899442b8944e416e
Jul 26 17:55:40.645440 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:40.645403    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-fdddb58cf-z4cng" event={"ID":"d382754b-8e98-4863-91ec-129ea9a09ac0","Type":"ContainerStarted","Data":"46c4dcd7f3e34b80284966a050442f60656ffad88ce7303a899442b8944e416e"}
Jul 26 17:55:41.654082 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:41.654035    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-fdddb58cf-z4cng" event={"ID":"d382754b-8e98-4863-91ec-129ea9a09ac0","Type":"ContainerStarted","Data":"54fa296e7017849fd1b8f02d94dce9cd3277ec9902c3abca7cde154c4647dfce"}
Jul 26 17:55:41.654878 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:41.654849    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-fdddb58cf-z4cng"
Jul 26 17:55:41.678038 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:41.677973    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-fdddb58cf-z4cng" podStartSLOduration=1.142820719 podStartE2EDuration="2.677952114s" totalImagesPullingTime="1.535131395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:55:41.674900917 +0000 UTC m=+398.327091777" watchObservedRunningTime="2026-07-26 17:55:41.677952114 +0000 UTC m=+398.330142974"
Jul 26 17:55:41.990726 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:41.990696    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"]
Jul 26 17:55:41.990934 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:41.990914    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4" podUID="118b15c4-0aa6-40b0-b550-2836c38d07c7" containerName="discovery" containerID="cri-o://7209ce1901ae6cf69446dccecf246f1c1b6585f09a902242fc31207e2748059f" gracePeriod=30
Jul 26 17:55:42.269778 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.269750    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:42.406281 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.406240    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/118b15c4-0aa6-40b0-b550-2836c38d07c7-cacerts\") pod \"118b15c4-0aa6-40b0-b550-2836c38d07c7\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") "
Jul 26 17:55:42.406281 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.406283    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/118b15c4-0aa6-40b0-b550-2836c38d07c7-istio-csr-dns-cert\") pod \"118b15c4-0aa6-40b0-b550-2836c38d07c7\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") "
Jul 26 17:55:42.406516 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.406310    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqrwh\" (UniqueName: \"kubernetes.io/projected/118b15c4-0aa6-40b0-b550-2836c38d07c7-kube-api-access-tqrwh\") pod \"118b15c4-0aa6-40b0-b550-2836c38d07c7\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") "
Jul 26 17:55:42.406516 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.406331    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/118b15c4-0aa6-40b0-b550-2836c38d07c7-istio-token\") pod \"118b15c4-0aa6-40b0-b550-2836c38d07c7\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") "
Jul 26 17:55:42.406516 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.406423    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/118b15c4-0aa6-40b0-b550-2836c38d07c7-local-certs\") pod \"118b15c4-0aa6-40b0-b550-2836c38d07c7\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") "
Jul 26 17:55:42.406516 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.406459    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/118b15c4-0aa6-40b0-b550-2836c38d07c7-istio-csr-ca-configmap\") pod \"118b15c4-0aa6-40b0-b550-2836c38d07c7\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") "
Jul 26 17:55:42.406516 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.406496    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/118b15c4-0aa6-40b0-b550-2836c38d07c7-istio-kubeconfig\") pod \"118b15c4-0aa6-40b0-b550-2836c38d07c7\" (UID: \"118b15c4-0aa6-40b0-b550-2836c38d07c7\") "
Jul 26 17:55:42.408657 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.408607    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/118b15c4-0aa6-40b0-b550-2836c38d07c7-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "118b15c4-0aa6-40b0-b550-2836c38d07c7" (UID: "118b15c4-0aa6-40b0-b550-2836c38d07c7"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:55:42.410582 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.410550    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/118b15c4-0aa6-40b0-b550-2836c38d07c7-istio-token" (OuterVolumeSpecName: "istio-token") pod "118b15c4-0aa6-40b0-b550-2836c38d07c7" (UID: "118b15c4-0aa6-40b0-b550-2836c38d07c7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:55:42.410704 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.410632    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/118b15c4-0aa6-40b0-b550-2836c38d07c7-local-certs" (OuterVolumeSpecName: "local-certs") pod "118b15c4-0aa6-40b0-b550-2836c38d07c7" (UID: "118b15c4-0aa6-40b0-b550-2836c38d07c7"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 17:55:42.410763 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.410698    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/118b15c4-0aa6-40b0-b550-2836c38d07c7-cacerts" (OuterVolumeSpecName: "cacerts") pod "118b15c4-0aa6-40b0-b550-2836c38d07c7" (UID: "118b15c4-0aa6-40b0-b550-2836c38d07c7"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 17:55:42.410763 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.410716    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/118b15c4-0aa6-40b0-b550-2836c38d07c7-kube-api-access-tqrwh" (OuterVolumeSpecName: "kube-api-access-tqrwh") pod "118b15c4-0aa6-40b0-b550-2836c38d07c7" (UID: "118b15c4-0aa6-40b0-b550-2836c38d07c7"). InnerVolumeSpecName "kube-api-access-tqrwh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:55:42.410844 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.410760    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/118b15c4-0aa6-40b0-b550-2836c38d07c7-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "118b15c4-0aa6-40b0-b550-2836c38d07c7" (UID: "118b15c4-0aa6-40b0-b550-2836c38d07c7"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 17:55:42.410907 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.410890    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/118b15c4-0aa6-40b0-b550-2836c38d07c7-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "118b15c4-0aa6-40b0-b550-2836c38d07c7" (UID: "118b15c4-0aa6-40b0-b550-2836c38d07c7"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 17:55:42.507565 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.507476    2570 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/118b15c4-0aa6-40b0-b550-2836c38d07c7-local-certs\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:55:42.507565 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.507509    2570 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/118b15c4-0aa6-40b0-b550-2836c38d07c7-istio-csr-ca-configmap\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:55:42.507565 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.507520    2570 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/118b15c4-0aa6-40b0-b550-2836c38d07c7-istio-kubeconfig\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:55:42.507565 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.507529    2570 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/118b15c4-0aa6-40b0-b550-2836c38d07c7-cacerts\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:55:42.507565 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.507537    2570 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/118b15c4-0aa6-40b0-b550-2836c38d07c7-istio-csr-dns-cert\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:55:42.507565 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.507545    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqrwh\" (UniqueName: \"kubernetes.io/projected/118b15c4-0aa6-40b0-b550-2836c38d07c7-kube-api-access-tqrwh\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:55:42.507565 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.507553    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/118b15c4-0aa6-40b0-b550-2836c38d07c7-istio-token\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:55:42.659694 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.659658    2570 generic.go:356] "Generic (PLEG): container finished" podID="118b15c4-0aa6-40b0-b550-2836c38d07c7" containerID="7209ce1901ae6cf69446dccecf246f1c1b6585f09a902242fc31207e2748059f" exitCode=0
Jul 26 17:55:42.660190 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.659711    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"
Jul 26 17:55:42.660190 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.659744    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4" event={"ID":"118b15c4-0aa6-40b0-b550-2836c38d07c7","Type":"ContainerDied","Data":"7209ce1901ae6cf69446dccecf246f1c1b6585f09a902242fc31207e2748059f"}
Jul 26 17:55:42.660190 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.659790    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4" event={"ID":"118b15c4-0aa6-40b0-b550-2836c38d07c7","Type":"ContainerDied","Data":"5a4295ca3f9804d428e1c61c6a75a093e926eba3fb5a6d27e97d778768fa7e52"}
Jul 26 17:55:42.660190 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.659810    2570 scope.go:117] "RemoveContainer" containerID="7209ce1901ae6cf69446dccecf246f1c1b6585f09a902242fc31207e2748059f"
Jul 26 17:55:42.668570 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.668554    2570 scope.go:117] "RemoveContainer" containerID="7209ce1901ae6cf69446dccecf246f1c1b6585f09a902242fc31207e2748059f"
Jul 26 17:55:42.668826 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:55:42.668807    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7209ce1901ae6cf69446dccecf246f1c1b6585f09a902242fc31207e2748059f\": container with ID starting with 7209ce1901ae6cf69446dccecf246f1c1b6585f09a902242fc31207e2748059f not found: ID does not exist" containerID="7209ce1901ae6cf69446dccecf246f1c1b6585f09a902242fc31207e2748059f"
Jul 26 17:55:42.668889 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.668838    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7209ce1901ae6cf69446dccecf246f1c1b6585f09a902242fc31207e2748059f"} err="failed to get container status \"7209ce1901ae6cf69446dccecf246f1c1b6585f09a902242fc31207e2748059f\": rpc error: code = NotFound desc = could not find container \"7209ce1901ae6cf69446dccecf246f1c1b6585f09a902242fc31207e2748059f\": container with ID starting with 7209ce1901ae6cf69446dccecf246f1c1b6585f09a902242fc31207e2748059f not found: ID does not exist"
Jul 26 17:55:42.692289 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.692267    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"]
Jul 26 17:55:42.698762 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:42.698739    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7sfr4"]
Jul 26 17:55:43.837230 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:43.837198    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="118b15c4-0aa6-40b0-b550-2836c38d07c7" path="/var/lib/kubelet/pods/118b15c4-0aa6-40b0-b550-2836c38d07c7/volumes"
Jul 26 17:55:46.627613 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:46.627583    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-jfjsd"
Jul 26 17:55:53.668352 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:55:53.668317    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-fdddb58cf-z4cng"
Jul 26 17:56:45.192647 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.192610    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh"]
Jul 26 17:56:45.193202 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.193082    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="118b15c4-0aa6-40b0-b550-2836c38d07c7" containerName="discovery"
Jul 26 17:56:45.193202 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.193102    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="118b15c4-0aa6-40b0-b550-2836c38d07c7" containerName="discovery"
Jul 26 17:56:45.193331 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.193210    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="118b15c4-0aa6-40b0-b550-2836c38d07c7" containerName="discovery"
Jul 26 17:56:45.196406 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.196387    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh"
Jul 26 17:56:45.199199 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.199172    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9sdhc\""
Jul 26 17:56:45.199199 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.199196    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 26 17:56:45.199351 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.199175    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 26 17:56:45.204720 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.204700    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh"]
Jul 26 17:56:45.335187 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.335152    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f237abf-e1b2-4587-9744-49fc265cd2f6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh\" (UID: \"5f237abf-e1b2-4587-9744-49fc265cd2f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh"
Jul 26 17:56:45.335367 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.335203    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f237abf-e1b2-4587-9744-49fc265cd2f6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh\" (UID: \"5f237abf-e1b2-4587-9744-49fc265cd2f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh"
Jul 26 17:56:45.335367 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.335233    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97qqc\" (UniqueName: \"kubernetes.io/projected/5f237abf-e1b2-4587-9744-49fc265cd2f6-kube-api-access-97qqc\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh\" (UID: \"5f237abf-e1b2-4587-9744-49fc265cd2f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh"
Jul 26 17:56:45.436642 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.436605    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f237abf-e1b2-4587-9744-49fc265cd2f6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh\" (UID: \"5f237abf-e1b2-4587-9744-49fc265cd2f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh"
Jul 26 17:56:45.436784 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.436654    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f237abf-e1b2-4587-9744-49fc265cd2f6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh\" (UID: \"5f237abf-e1b2-4587-9744-49fc265cd2f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh"
Jul 26 17:56:45.436784 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.436679    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97qqc\" (UniqueName: \"kubernetes.io/projected/5f237abf-e1b2-4587-9744-49fc265cd2f6-kube-api-access-97qqc\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh\" (UID: \"5f237abf-e1b2-4587-9744-49fc265cd2f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh"
Jul 26 17:56:45.437015 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.436995    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f237abf-e1b2-4587-9744-49fc265cd2f6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh\" (UID: \"5f237abf-e1b2-4587-9744-49fc265cd2f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh"
Jul 26 17:56:45.437052 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.437026    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f237abf-e1b2-4587-9744-49fc265cd2f6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh\" (UID: \"5f237abf-e1b2-4587-9744-49fc265cd2f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh"
Jul 26 17:56:45.446219 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.446152    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97qqc\" (UniqueName: \"kubernetes.io/projected/5f237abf-e1b2-4587-9744-49fc265cd2f6-kube-api-access-97qqc\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh\" (UID: \"5f237abf-e1b2-4587-9744-49fc265cd2f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh"
Jul 26 17:56:45.506948 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.506911    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh"
Jul 26 17:56:45.646973 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.646928    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh"]
Jul 26 17:56:45.653167 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:56:45.653138    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f237abf_e1b2_4587_9744_49fc265cd2f6.slice/crio-21e7ecd4232a06bdbd385447b47e1d10026ef2e668ae162e287be4e12d3a151e WatchSource:0}: Error finding container 21e7ecd4232a06bdbd385447b47e1d10026ef2e668ae162e287be4e12d3a151e: Status 404 returned error can't find the container with id 21e7ecd4232a06bdbd385447b47e1d10026ef2e668ae162e287be4e12d3a151e
Jul 26 17:56:45.784005 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.783969    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9"]
Jul 26 17:56:45.787500 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.787482    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9"
Jul 26 17:56:45.795581 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.795554    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9"]
Jul 26 17:56:45.942063 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.942020    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/168868e8-b9f1-483a-8914-779244f7f85e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9\" (UID: \"168868e8-b9f1-483a-8914-779244f7f85e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9"
Jul 26 17:56:45.942063 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.942065    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxbjz\" (UniqueName: \"kubernetes.io/projected/168868e8-b9f1-483a-8914-779244f7f85e-kube-api-access-zxbjz\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9\" (UID: \"168868e8-b9f1-483a-8914-779244f7f85e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9"
Jul 26 17:56:45.942347 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.942198    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/168868e8-b9f1-483a-8914-779244f7f85e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9\" (UID: \"168868e8-b9f1-483a-8914-779244f7f85e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9"
Jul 26 17:56:45.963666 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.963587    2570 generic.go:356] "Generic (PLEG): container finished" podID="5f237abf-e1b2-4587-9744-49fc265cd2f6" containerID="d7d9273f746bb86c2530e5a46f28922fe13c4260c0513f38e7c81077a5cf0420" exitCode=0
Jul 26 17:56:45.963805 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.963668    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh" event={"ID":"5f237abf-e1b2-4587-9744-49fc265cd2f6","Type":"ContainerDied","Data":"d7d9273f746bb86c2530e5a46f28922fe13c4260c0513f38e7c81077a5cf0420"}
Jul 26 17:56:45.963805 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:45.963723    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh" event={"ID":"5f237abf-e1b2-4587-9744-49fc265cd2f6","Type":"ContainerStarted","Data":"21e7ecd4232a06bdbd385447b47e1d10026ef2e668ae162e287be4e12d3a151e"}
Jul 26 17:56:46.042868 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.042823    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/168868e8-b9f1-483a-8914-779244f7f85e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9\" (UID: \"168868e8-b9f1-483a-8914-779244f7f85e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9"
Jul 26 17:56:46.043067 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.042917    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/168868e8-b9f1-483a-8914-779244f7f85e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9\" (UID: \"168868e8-b9f1-483a-8914-779244f7f85e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9"
Jul 26 17:56:46.043067 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.042944    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxbjz\" (UniqueName: \"kubernetes.io/projected/168868e8-b9f1-483a-8914-779244f7f85e-kube-api-access-zxbjz\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9\" (UID: \"168868e8-b9f1-483a-8914-779244f7f85e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9"
Jul 26 17:56:46.043274 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.043253    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/168868e8-b9f1-483a-8914-779244f7f85e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9\" (UID: \"168868e8-b9f1-483a-8914-779244f7f85e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9"
Jul 26 17:56:46.043336 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.043289    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/168868e8-b9f1-483a-8914-779244f7f85e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9\" (UID: \"168868e8-b9f1-483a-8914-779244f7f85e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9"
Jul 26 17:56:46.050991 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.050963    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxbjz\" (UniqueName: \"kubernetes.io/projected/168868e8-b9f1-483a-8914-779244f7f85e-kube-api-access-zxbjz\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9\" (UID: \"168868e8-b9f1-483a-8914-779244f7f85e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9"
Jul 26 17:56:46.123108 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.123057    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9"
Jul 26 17:56:46.250855 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.250825    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9"]
Jul 26 17:56:46.252433 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:56:46.252410    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod168868e8_b9f1_483a_8914_779244f7f85e.slice/crio-37224a413f05364c3079b0d16af0393481705fa7c7f83c7ab160a6e574621fde WatchSource:0}: Error finding container 37224a413f05364c3079b0d16af0393481705fa7c7f83c7ab160a6e574621fde: Status 404 returned error can't find the container with id 37224a413f05364c3079b0d16af0393481705fa7c7f83c7ab160a6e574621fde
Jul 26 17:56:46.390725 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.390694    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2"]
Jul 26 17:56:46.393930 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.393912    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2"
Jul 26 17:56:46.402149 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.402105    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2"]
Jul 26 17:56:46.548291 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.548258    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9ff4b55-0d26-4800-91ce-ed80d3468084-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2\" (UID: \"c9ff4b55-0d26-4800-91ce-ed80d3468084\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2"
Jul 26 17:56:46.548473 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.548337    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9ff4b55-0d26-4800-91ce-ed80d3468084-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2\" (UID: \"c9ff4b55-0d26-4800-91ce-ed80d3468084\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2"
Jul 26 17:56:46.548473 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.548361    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szpgk\" (UniqueName: \"kubernetes.io/projected/c9ff4b55-0d26-4800-91ce-ed80d3468084-kube-api-access-szpgk\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2\" (UID: \"c9ff4b55-0d26-4800-91ce-ed80d3468084\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2"
Jul 26 17:56:46.649372 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.649337    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9ff4b55-0d26-4800-91ce-ed80d3468084-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2\" (UID: \"c9ff4b55-0d26-4800-91ce-ed80d3468084\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2"
Jul 26 17:56:46.649558 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.649437    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9ff4b55-0d26-4800-91ce-ed80d3468084-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2\" (UID: \"c9ff4b55-0d26-4800-91ce-ed80d3468084\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2"
Jul 26 17:56:46.649558 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.649466    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szpgk\" (UniqueName: \"kubernetes.io/projected/c9ff4b55-0d26-4800-91ce-ed80d3468084-kube-api-access-szpgk\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2\" (UID: \"c9ff4b55-0d26-4800-91ce-ed80d3468084\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2"
Jul 26 17:56:46.649735 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.649713    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9ff4b55-0d26-4800-91ce-ed80d3468084-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2\" (UID: \"c9ff4b55-0d26-4800-91ce-ed80d3468084\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2"
Jul 26 17:56:46.649797 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.649750    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9ff4b55-0d26-4800-91ce-ed80d3468084-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2\" (UID: \"c9ff4b55-0d26-4800-91ce-ed80d3468084\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2"
Jul 26 17:56:46.657915 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.657879    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szpgk\" (UniqueName: \"kubernetes.io/projected/c9ff4b55-0d26-4800-91ce-ed80d3468084-kube-api-access-szpgk\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2\" (UID: \"c9ff4b55-0d26-4800-91ce-ed80d3468084\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2"
Jul 26 17:56:46.708465 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.708428    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2"
Jul 26 17:56:46.793049 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.793025    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn"]
Jul 26 17:56:46.796881 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.796853    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn"
Jul 26 17:56:46.809526 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.809500    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn"]
Jul 26 17:56:46.853519 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.853464    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2"]
Jul 26 17:56:46.876021 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:56:46.875982    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9ff4b55_0d26_4800_91ce_ed80d3468084.slice/crio-025eba583170788a30990d29c67bf42bc9707fd06a78a3d2ca63ee236681a154 WatchSource:0}: Error finding container 025eba583170788a30990d29c67bf42bc9707fd06a78a3d2ca63ee236681a154: Status 404 returned error can't find the container with id 025eba583170788a30990d29c67bf42bc9707fd06a78a3d2ca63ee236681a154
Jul 26 17:56:46.952469 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.952430    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj7x5\" (UniqueName: \"kubernetes.io/projected/ec4e20d8-f65b-42e6-b43b-ca57ebefe970-kube-api-access-nj7x5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn\" (UID: \"ec4e20d8-f65b-42e6-b43b-ca57ebefe970\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn"
Jul 26 17:56:46.952640 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.952504    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec4e20d8-f65b-42e6-b43b-ca57ebefe970-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn\" (UID: \"ec4e20d8-f65b-42e6-b43b-ca57ebefe970\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn"
Jul 26 17:56:46.952640 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.952531    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec4e20d8-f65b-42e6-b43b-ca57ebefe970-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn\" (UID: \"ec4e20d8-f65b-42e6-b43b-ca57ebefe970\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn"
Jul 26 17:56:46.971239 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.971208    2570 generic.go:356] "Generic (PLEG): container finished" podID="168868e8-b9f1-483a-8914-779244f7f85e" containerID="b5888a11f9b02b75740a882a8e473d163bc19ae73d027d7a1457aeee757ad8c8" exitCode=0
Jul 26 17:56:46.971348 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.971330    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9" event={"ID":"168868e8-b9f1-483a-8914-779244f7f85e","Type":"ContainerDied","Data":"b5888a11f9b02b75740a882a8e473d163bc19ae73d027d7a1457aeee757ad8c8"}
Jul 26 17:56:46.971421 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.971382    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9" event={"ID":"168868e8-b9f1-483a-8914-779244f7f85e","Type":"ContainerStarted","Data":"37224a413f05364c3079b0d16af0393481705fa7c7f83c7ab160a6e574621fde"}
Jul 26 17:56:46.973435 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.973406    2570 generic.go:356] "Generic (PLEG): container finished" podID="5f237abf-e1b2-4587-9744-49fc265cd2f6" containerID="057107cabc90cf5b5489957948e6155d8c457d169606c6cacc569912d60f1ac6" exitCode=0
Jul 26 17:56:46.973546 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.973491    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh" event={"ID":"5f237abf-e1b2-4587-9744-49fc265cd2f6","Type":"ContainerDied","Data":"057107cabc90cf5b5489957948e6155d8c457d169606c6cacc569912d60f1ac6"}
Jul 26 17:56:46.979051 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.979021    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2" event={"ID":"c9ff4b55-0d26-4800-91ce-ed80d3468084","Type":"ContainerStarted","Data":"b79d21ed7db84b33b65728adaeb1174310889a8439fd981efe0cdeef6f9c5659"}
Jul 26 17:56:46.979174 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:46.979072    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2" event={"ID":"c9ff4b55-0d26-4800-91ce-ed80d3468084","Type":"ContainerStarted","Data":"025eba583170788a30990d29c67bf42bc9707fd06a78a3d2ca63ee236681a154"}
Jul 26 17:56:47.053604 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:47.053575    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec4e20d8-f65b-42e6-b43b-ca57ebefe970-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn\" (UID: \"ec4e20d8-f65b-42e6-b43b-ca57ebefe970\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn"
Jul 26 17:56:47.053604 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:47.053610    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec4e20d8-f65b-42e6-b43b-ca57ebefe970-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn\" (UID: \"ec4e20d8-f65b-42e6-b43b-ca57ebefe970\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn"
Jul 26 17:56:47.053796 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:47.053673    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj7x5\" (UniqueName: \"kubernetes.io/projected/ec4e20d8-f65b-42e6-b43b-ca57ebefe970-kube-api-access-nj7x5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn\" (UID: \"ec4e20d8-f65b-42e6-b43b-ca57ebefe970\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn"
Jul 26 17:56:47.054050 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:47.054024    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec4e20d8-f65b-42e6-b43b-ca57ebefe970-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn\" (UID: \"ec4e20d8-f65b-42e6-b43b-ca57ebefe970\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn"
Jul 26 17:56:47.054095 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:47.054077    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec4e20d8-f65b-42e6-b43b-ca57ebefe970-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn\" (UID: \"ec4e20d8-f65b-42e6-b43b-ca57ebefe970\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn"
Jul 26 17:56:47.062581 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:47.062508    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj7x5\" (UniqueName: \"kubernetes.io/projected/ec4e20d8-f65b-42e6-b43b-ca57ebefe970-kube-api-access-nj7x5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn\" (UID: \"ec4e20d8-f65b-42e6-b43b-ca57ebefe970\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn"
Jul 26 17:56:47.110990 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:47.110964    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn"
Jul 26 17:56:47.244856 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:47.244824    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn"]
Jul 26 17:56:47.246476 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:56:47.246454    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec4e20d8_f65b_42e6_b43b_ca57ebefe970.slice/crio-7a51451a152f97513cbf78230ca5d736becbfef8e8fad097513782b189020913 WatchSource:0}: Error finding container 7a51451a152f97513cbf78230ca5d736becbfef8e8fad097513782b189020913: Status 404 returned error can't find the container with id 7a51451a152f97513cbf78230ca5d736becbfef8e8fad097513782b189020913
Jul 26 17:56:47.985731 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:47.985694    2570 generic.go:356] "Generic (PLEG): container finished" podID="5f237abf-e1b2-4587-9744-49fc265cd2f6" containerID="104e038459eb7bdf2cb104b551fd0e27eedb496880d915f033591b9d3ae696ee" exitCode=0
Jul 26 17:56:47.986144 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:47.985777    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh" event={"ID":"5f237abf-e1b2-4587-9744-49fc265cd2f6","Type":"ContainerDied","Data":"104e038459eb7bdf2cb104b551fd0e27eedb496880d915f033591b9d3ae696ee"}
Jul 26 17:56:47.987232 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:47.987205    2570 generic.go:356] "Generic (PLEG): container finished" podID="c9ff4b55-0d26-4800-91ce-ed80d3468084" containerID="b79d21ed7db84b33b65728adaeb1174310889a8439fd981efe0cdeef6f9c5659" exitCode=0
Jul 26 17:56:47.987315 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:47.987267    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2" event={"ID":"c9ff4b55-0d26-4800-91ce-ed80d3468084","Type":"ContainerDied","Data":"b79d21ed7db84b33b65728adaeb1174310889a8439fd981efe0cdeef6f9c5659"}
Jul 26 17:56:47.989181 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:47.989162    2570 generic.go:356] "Generic (PLEG): container finished" podID="168868e8-b9f1-483a-8914-779244f7f85e" containerID="656d04c57a4c76f1fcdef866bb62a69bc25ba595858e1561f555d4f19ed4523e" exitCode=0
Jul 26 17:56:47.989270 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:47.989188    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9" event={"ID":"168868e8-b9f1-483a-8914-779244f7f85e","Type":"ContainerDied","Data":"656d04c57a4c76f1fcdef866bb62a69bc25ba595858e1561f555d4f19ed4523e"}
Jul 26 17:56:47.990970 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:47.990947    2570 generic.go:356] "Generic (PLEG): container finished" podID="ec4e20d8-f65b-42e6-b43b-ca57ebefe970" containerID="74b1389ee3226325fd6d5b3aa71d7d700dbca2ed67faeb6bdf5658e404027bc7" exitCode=0
Jul 26 17:56:47.991236 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:47.990979    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn" event={"ID":"ec4e20d8-f65b-42e6-b43b-ca57ebefe970","Type":"ContainerDied","Data":"74b1389ee3226325fd6d5b3aa71d7d700dbca2ed67faeb6bdf5658e404027bc7"}
Jul 26 17:56:47.991236 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:47.991004    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn" event={"ID":"ec4e20d8-f65b-42e6-b43b-ca57ebefe970","Type":"ContainerStarted","Data":"7a51451a152f97513cbf78230ca5d736becbfef8e8fad097513782b189020913"}
Jul 26 17:56:48.999302 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:48.999271    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn" event={"ID":"ec4e20d8-f65b-42e6-b43b-ca57ebefe970","Type":"ContainerStarted","Data":"21ca9056b05836a03184ac42c9601e9ce06b6bf3dca0824bbccdf6ba89e36d16"}
Jul 26 17:56:49.001289 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:49.001265    2570 generic.go:356] "Generic (PLEG): container finished" podID="c9ff4b55-0d26-4800-91ce-ed80d3468084" containerID="e8182b0378e13208b8b81861a0a84c7de27261c5c324307e04e3cd64c3203088" exitCode=0
Jul 26 17:56:49.001413 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:49.001340    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2" event={"ID":"c9ff4b55-0d26-4800-91ce-ed80d3468084","Type":"ContainerDied","Data":"e8182b0378e13208b8b81861a0a84c7de27261c5c324307e04e3cd64c3203088"}
Jul 26 17:56:49.003649 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:49.003622    2570 generic.go:356] "Generic (PLEG): container finished" podID="168868e8-b9f1-483a-8914-779244f7f85e" containerID="618842e8ca6caf5b1189ab608bbaeeaf2c842715abe9fc99f427f1267d83b0fa" exitCode=0
Jul 26 17:56:49.003790 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:49.003698    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9" event={"ID":"168868e8-b9f1-483a-8914-779244f7f85e","Type":"ContainerDied","Data":"618842e8ca6caf5b1189ab608bbaeeaf2c842715abe9fc99f427f1267d83b0fa"}
Jul 26 17:56:49.127234 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:49.127214    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh"
Jul 26 17:56:49.169658 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:49.169625    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f237abf-e1b2-4587-9744-49fc265cd2f6-bundle\") pod \"5f237abf-e1b2-4587-9744-49fc265cd2f6\" (UID: \"5f237abf-e1b2-4587-9744-49fc265cd2f6\") "
Jul 26 17:56:49.169792 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:49.169699    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f237abf-e1b2-4587-9744-49fc265cd2f6-util\") pod \"5f237abf-e1b2-4587-9744-49fc265cd2f6\" (UID: \"5f237abf-e1b2-4587-9744-49fc265cd2f6\") "
Jul 26 17:56:49.169792 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:49.169751    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97qqc\" (UniqueName: \"kubernetes.io/projected/5f237abf-e1b2-4587-9744-49fc265cd2f6-kube-api-access-97qqc\") pod \"5f237abf-e1b2-4587-9744-49fc265cd2f6\" (UID: \"5f237abf-e1b2-4587-9744-49fc265cd2f6\") "
Jul 26 17:56:49.170519 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:49.170330    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f237abf-e1b2-4587-9744-49fc265cd2f6-bundle" (OuterVolumeSpecName: "bundle") pod "5f237abf-e1b2-4587-9744-49fc265cd2f6" (UID: "5f237abf-e1b2-4587-9744-49fc265cd2f6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 17:56:49.170519 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:49.170493    2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f237abf-e1b2-4587-9744-49fc265cd2f6-bundle\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:56:49.172464 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:49.172439    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f237abf-e1b2-4587-9744-49fc265cd2f6-kube-api-access-97qqc" (OuterVolumeSpecName: "kube-api-access-97qqc") pod "5f237abf-e1b2-4587-9744-49fc265cd2f6" (UID: "5f237abf-e1b2-4587-9744-49fc265cd2f6"). InnerVolumeSpecName "kube-api-access-97qqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:56:49.175866 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:49.175844    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f237abf-e1b2-4587-9744-49fc265cd2f6-util" (OuterVolumeSpecName: "util") pod "5f237abf-e1b2-4587-9744-49fc265cd2f6" (UID: "5f237abf-e1b2-4587-9744-49fc265cd2f6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 17:56:49.271282 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:49.271247    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97qqc\" (UniqueName: \"kubernetes.io/projected/5f237abf-e1b2-4587-9744-49fc265cd2f6-kube-api-access-97qqc\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:56:49.271282 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:49.271279    2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f237abf-e1b2-4587-9744-49fc265cd2f6-util\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:56:50.010334 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:50.010289    2570 generic.go:356] "Generic (PLEG): container finished" podID="c9ff4b55-0d26-4800-91ce-ed80d3468084" containerID="2f85e87e71226340943b11bf983536453ef94e5ba8339dab661a28e6727c9c5c" exitCode=0
Jul 26 17:56:50.010334 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:50.010327    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2" event={"ID":"c9ff4b55-0d26-4800-91ce-ed80d3468084","Type":"ContainerDied","Data":"2f85e87e71226340943b11bf983536453ef94e5ba8339dab661a28e6727c9c5c"}
Jul 26 17:56:50.011999 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:50.011974    2570 generic.go:356] "Generic (PLEG): container finished" podID="ec4e20d8-f65b-42e6-b43b-ca57ebefe970" containerID="21ca9056b05836a03184ac42c9601e9ce06b6bf3dca0824bbccdf6ba89e36d16" exitCode=0
Jul 26 17:56:50.012145 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:50.012012    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn" event={"ID":"ec4e20d8-f65b-42e6-b43b-ca57ebefe970","Type":"ContainerDied","Data":"21ca9056b05836a03184ac42c9601e9ce06b6bf3dca0824bbccdf6ba89e36d16"}
Jul 26 17:56:50.014144 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:50.014108    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh" event={"ID":"5f237abf-e1b2-4587-9744-49fc265cd2f6","Type":"ContainerDied","Data":"21e7ecd4232a06bdbd385447b47e1d10026ef2e668ae162e287be4e12d3a151e"}
Jul 26 17:56:50.014222 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:50.014181    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="21e7ecd4232a06bdbd385447b47e1d10026ef2e668ae162e287be4e12d3a151e"
Jul 26 17:56:50.014343 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:50.014329    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh"
Jul 26 17:56:50.148810 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:50.148787    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9"
Jul 26 17:56:50.179407 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:50.179382    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/168868e8-b9f1-483a-8914-779244f7f85e-util\") pod \"168868e8-b9f1-483a-8914-779244f7f85e\" (UID: \"168868e8-b9f1-483a-8914-779244f7f85e\") "
Jul 26 17:56:50.179531 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:50.179434    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/168868e8-b9f1-483a-8914-779244f7f85e-bundle\") pod \"168868e8-b9f1-483a-8914-779244f7f85e\" (UID: \"168868e8-b9f1-483a-8914-779244f7f85e\") "
Jul 26 17:56:50.179531 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:50.179461    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxbjz\" (UniqueName: \"kubernetes.io/projected/168868e8-b9f1-483a-8914-779244f7f85e-kube-api-access-zxbjz\") pod \"168868e8-b9f1-483a-8914-779244f7f85e\" (UID: \"168868e8-b9f1-483a-8914-779244f7f85e\") "
Jul 26 17:56:50.179996 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:50.179970    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/168868e8-b9f1-483a-8914-779244f7f85e-bundle" (OuterVolumeSpecName: "bundle") pod "168868e8-b9f1-483a-8914-779244f7f85e" (UID: "168868e8-b9f1-483a-8914-779244f7f85e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 17:56:50.181454 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:50.181432    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/168868e8-b9f1-483a-8914-779244f7f85e-kube-api-access-zxbjz" (OuterVolumeSpecName: "kube-api-access-zxbjz") pod "168868e8-b9f1-483a-8914-779244f7f85e" (UID: "168868e8-b9f1-483a-8914-779244f7f85e"). InnerVolumeSpecName "kube-api-access-zxbjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:56:50.185379 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:50.185354    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/168868e8-b9f1-483a-8914-779244f7f85e-util" (OuterVolumeSpecName: "util") pod "168868e8-b9f1-483a-8914-779244f7f85e" (UID: "168868e8-b9f1-483a-8914-779244f7f85e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 17:56:50.280200 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:50.280170    2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/168868e8-b9f1-483a-8914-779244f7f85e-util\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:56:50.280200 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:50.280195    2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/168868e8-b9f1-483a-8914-779244f7f85e-bundle\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:56:50.280200 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:50.280206    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxbjz\" (UniqueName: \"kubernetes.io/projected/168868e8-b9f1-483a-8914-779244f7f85e-kube-api-access-zxbjz\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:56:51.021298 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:51.021269    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9"
Jul 26 17:56:51.021669 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:51.021292    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9" event={"ID":"168868e8-b9f1-483a-8914-779244f7f85e","Type":"ContainerDied","Data":"37224a413f05364c3079b0d16af0393481705fa7c7f83c7ab160a6e574621fde"}
Jul 26 17:56:51.021669 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:51.021339    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="37224a413f05364c3079b0d16af0393481705fa7c7f83c7ab160a6e574621fde"
Jul 26 17:56:51.023442 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:51.023419    2570 generic.go:356] "Generic (PLEG): container finished" podID="ec4e20d8-f65b-42e6-b43b-ca57ebefe970" containerID="b88ac3f88f1ed502075845b1b26e6e11bcd915702394e1070175910573411371" exitCode=0
Jul 26 17:56:51.023564 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:51.023479    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn" event={"ID":"ec4e20d8-f65b-42e6-b43b-ca57ebefe970","Type":"ContainerDied","Data":"b88ac3f88f1ed502075845b1b26e6e11bcd915702394e1070175910573411371"}
Jul 26 17:56:51.149611 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:51.149585    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2"
Jul 26 17:56:51.194775 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:51.194743    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9ff4b55-0d26-4800-91ce-ed80d3468084-util\") pod \"c9ff4b55-0d26-4800-91ce-ed80d3468084\" (UID: \"c9ff4b55-0d26-4800-91ce-ed80d3468084\") "
Jul 26 17:56:51.194960 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:51.194799    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9ff4b55-0d26-4800-91ce-ed80d3468084-bundle\") pod \"c9ff4b55-0d26-4800-91ce-ed80d3468084\" (UID: \"c9ff4b55-0d26-4800-91ce-ed80d3468084\") "
Jul 26 17:56:51.194960 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:51.194829    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szpgk\" (UniqueName: \"kubernetes.io/projected/c9ff4b55-0d26-4800-91ce-ed80d3468084-kube-api-access-szpgk\") pod \"c9ff4b55-0d26-4800-91ce-ed80d3468084\" (UID: \"c9ff4b55-0d26-4800-91ce-ed80d3468084\") "
Jul 26 17:56:51.195336 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:51.195309    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9ff4b55-0d26-4800-91ce-ed80d3468084-bundle" (OuterVolumeSpecName: "bundle") pod "c9ff4b55-0d26-4800-91ce-ed80d3468084" (UID: "c9ff4b55-0d26-4800-91ce-ed80d3468084"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 17:56:51.196963 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:51.196930    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9ff4b55-0d26-4800-91ce-ed80d3468084-kube-api-access-szpgk" (OuterVolumeSpecName: "kube-api-access-szpgk") pod "c9ff4b55-0d26-4800-91ce-ed80d3468084" (UID: "c9ff4b55-0d26-4800-91ce-ed80d3468084"). InnerVolumeSpecName "kube-api-access-szpgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:56:51.200304 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:51.200279    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9ff4b55-0d26-4800-91ce-ed80d3468084-util" (OuterVolumeSpecName: "util") pod "c9ff4b55-0d26-4800-91ce-ed80d3468084" (UID: "c9ff4b55-0d26-4800-91ce-ed80d3468084"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 17:56:51.295657 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:51.295629    2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9ff4b55-0d26-4800-91ce-ed80d3468084-util\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:56:51.295657 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:51.295652    2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9ff4b55-0d26-4800-91ce-ed80d3468084-bundle\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:56:51.295657 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:51.295661    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szpgk\" (UniqueName: \"kubernetes.io/projected/c9ff4b55-0d26-4800-91ce-ed80d3468084-kube-api-access-szpgk\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:56:52.030448 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:52.030419    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2"
Jul 26 17:56:52.030813 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:52.030449    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2" event={"ID":"c9ff4b55-0d26-4800-91ce-ed80d3468084","Type":"ContainerDied","Data":"025eba583170788a30990d29c67bf42bc9707fd06a78a3d2ca63ee236681a154"}
Jul 26 17:56:52.030813 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:52.030501    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="025eba583170788a30990d29c67bf42bc9707fd06a78a3d2ca63ee236681a154"
Jul 26 17:56:52.152367 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:52.152341    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn"
Jul 26 17:56:52.203613 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:52.203586    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec4e20d8-f65b-42e6-b43b-ca57ebefe970-util\") pod \"ec4e20d8-f65b-42e6-b43b-ca57ebefe970\" (UID: \"ec4e20d8-f65b-42e6-b43b-ca57ebefe970\") "
Jul 26 17:56:52.203787 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:52.203630    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec4e20d8-f65b-42e6-b43b-ca57ebefe970-bundle\") pod \"ec4e20d8-f65b-42e6-b43b-ca57ebefe970\" (UID: \"ec4e20d8-f65b-42e6-b43b-ca57ebefe970\") "
Jul 26 17:56:52.203787 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:52.203669    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nj7x5\" (UniqueName: \"kubernetes.io/projected/ec4e20d8-f65b-42e6-b43b-ca57ebefe970-kube-api-access-nj7x5\") pod \"ec4e20d8-f65b-42e6-b43b-ca57ebefe970\" (UID: \"ec4e20d8-f65b-42e6-b43b-ca57ebefe970\") "
Jul 26 17:56:52.204146 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:52.204101    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec4e20d8-f65b-42e6-b43b-ca57ebefe970-bundle" (OuterVolumeSpecName: "bundle") pod "ec4e20d8-f65b-42e6-b43b-ca57ebefe970" (UID: "ec4e20d8-f65b-42e6-b43b-ca57ebefe970"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 17:56:52.205800 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:52.205768    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec4e20d8-f65b-42e6-b43b-ca57ebefe970-kube-api-access-nj7x5" (OuterVolumeSpecName: "kube-api-access-nj7x5") pod "ec4e20d8-f65b-42e6-b43b-ca57ebefe970" (UID: "ec4e20d8-f65b-42e6-b43b-ca57ebefe970"). InnerVolumeSpecName "kube-api-access-nj7x5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:56:52.209151 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:52.209107    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec4e20d8-f65b-42e6-b43b-ca57ebefe970-util" (OuterVolumeSpecName: "util") pod "ec4e20d8-f65b-42e6-b43b-ca57ebefe970" (UID: "ec4e20d8-f65b-42e6-b43b-ca57ebefe970"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 17:56:52.304399 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:52.304363    2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec4e20d8-f65b-42e6-b43b-ca57ebefe970-util\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:56:52.304399 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:52.304393    2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec4e20d8-f65b-42e6-b43b-ca57ebefe970-bundle\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:56:52.304399 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:52.304405    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nj7x5\" (UniqueName: \"kubernetes.io/projected/ec4e20d8-f65b-42e6-b43b-ca57ebefe970-kube-api-access-nj7x5\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:56:53.037031 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:53.036998    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn"
Jul 26 17:56:53.037467 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:53.037029    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn" event={"ID":"ec4e20d8-f65b-42e6-b43b-ca57ebefe970","Type":"ContainerDied","Data":"7a51451a152f97513cbf78230ca5d736becbfef8e8fad097513782b189020913"}
Jul 26 17:56:53.037467 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:56:53.037080    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7a51451a152f97513cbf78230ca5d736becbfef8e8fad097513782b189020913"
Jul 26 17:57:03.490010 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.489976    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84476cb65f-d4sm9"]
Jul 26 17:57:03.490485 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490339    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="168868e8-b9f1-483a-8914-779244f7f85e" containerName="util"
Jul 26 17:57:03.490485 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490352    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="168868e8-b9f1-483a-8914-779244f7f85e" containerName="util"
Jul 26 17:57:03.490485 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490361    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9ff4b55-0d26-4800-91ce-ed80d3468084" containerName="util"
Jul 26 17:57:03.490485 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490366    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9ff4b55-0d26-4800-91ce-ed80d3468084" containerName="util"
Jul 26 17:57:03.490485 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490375    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec4e20d8-f65b-42e6-b43b-ca57ebefe970" containerName="util"
Jul 26 17:57:03.490485 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490380    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec4e20d8-f65b-42e6-b43b-ca57ebefe970" containerName="util"
Jul 26 17:57:03.490485 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490387    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec4e20d8-f65b-42e6-b43b-ca57ebefe970" containerName="pull"
Jul 26 17:57:03.490485 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490393    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec4e20d8-f65b-42e6-b43b-ca57ebefe970" containerName="pull"
Jul 26 17:57:03.490485 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490407    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec4e20d8-f65b-42e6-b43b-ca57ebefe970" containerName="extract"
Jul 26 17:57:03.490485 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490413    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec4e20d8-f65b-42e6-b43b-ca57ebefe970" containerName="extract"
Jul 26 17:57:03.490485 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490421    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9ff4b55-0d26-4800-91ce-ed80d3468084" containerName="extract"
Jul 26 17:57:03.490485 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490425    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9ff4b55-0d26-4800-91ce-ed80d3468084" containerName="extract"
Jul 26 17:57:03.490485 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490435    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="168868e8-b9f1-483a-8914-779244f7f85e" containerName="extract"
Jul 26 17:57:03.490485 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490440    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="168868e8-b9f1-483a-8914-779244f7f85e" containerName="extract"
Jul 26 17:57:03.490485 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490449    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="168868e8-b9f1-483a-8914-779244f7f85e" containerName="pull"
Jul 26 17:57:03.490485 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490457    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="168868e8-b9f1-483a-8914-779244f7f85e" containerName="pull"
Jul 26 17:57:03.490485 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490482    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f237abf-e1b2-4587-9744-49fc265cd2f6" containerName="pull"
Jul 26 17:57:03.490485 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490491    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f237abf-e1b2-4587-9744-49fc265cd2f6" containerName="pull"
Jul 26 17:57:03.491661 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490507    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9ff4b55-0d26-4800-91ce-ed80d3468084" containerName="pull"
Jul 26 17:57:03.491661 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490529    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9ff4b55-0d26-4800-91ce-ed80d3468084" containerName="pull"
Jul 26 17:57:03.491661 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490544    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f237abf-e1b2-4587-9744-49fc265cd2f6" containerName="util"
Jul 26 17:57:03.491661 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490552    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f237abf-e1b2-4587-9744-49fc265cd2f6" containerName="util"
Jul 26 17:57:03.491661 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490561    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f237abf-e1b2-4587-9744-49fc265cd2f6" containerName="extract"
Jul 26 17:57:03.491661 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490575    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f237abf-e1b2-4587-9744-49fc265cd2f6" containerName="extract"
Jul 26 17:57:03.491661 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490664    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9ff4b55-0d26-4800-91ce-ed80d3468084" containerName="extract"
Jul 26 17:57:03.491661 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490675    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="168868e8-b9f1-483a-8914-779244f7f85e" containerName="extract"
Jul 26 17:57:03.491661 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490685    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f237abf-e1b2-4587-9744-49fc265cd2f6" containerName="extract"
Jul 26 17:57:03.491661 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.490698    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec4e20d8-f65b-42e6-b43b-ca57ebefe970" containerName="extract"
Jul 26 17:57:03.493947 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.493924    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:03.504843 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.504820    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84476cb65f-d4sm9"]
Jul 26 17:57:03.592526 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.592491    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72b366d6-ec21-49f4-b656-a07aed1f25c7-trusted-ca-bundle\") pod \"console-84476cb65f-d4sm9\" (UID: \"72b366d6-ec21-49f4-b656-a07aed1f25c7\") " pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:03.592709 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.592536    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/72b366d6-ec21-49f4-b656-a07aed1f25c7-console-serving-cert\") pod \"console-84476cb65f-d4sm9\" (UID: \"72b366d6-ec21-49f4-b656-a07aed1f25c7\") " pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:03.592709 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.592602    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/72b366d6-ec21-49f4-b656-a07aed1f25c7-oauth-serving-cert\") pod \"console-84476cb65f-d4sm9\" (UID: \"72b366d6-ec21-49f4-b656-a07aed1f25c7\") " pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:03.592709 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.592654    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/72b366d6-ec21-49f4-b656-a07aed1f25c7-service-ca\") pod \"console-84476cb65f-d4sm9\" (UID: \"72b366d6-ec21-49f4-b656-a07aed1f25c7\") " pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:03.592709 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.592674    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psjr4\" (UniqueName: \"kubernetes.io/projected/72b366d6-ec21-49f4-b656-a07aed1f25c7-kube-api-access-psjr4\") pod \"console-84476cb65f-d4sm9\" (UID: \"72b366d6-ec21-49f4-b656-a07aed1f25c7\") " pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:03.592709 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.592693    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/72b366d6-ec21-49f4-b656-a07aed1f25c7-console-config\") pod \"console-84476cb65f-d4sm9\" (UID: \"72b366d6-ec21-49f4-b656-a07aed1f25c7\") " pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:03.592867 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.592735    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/72b366d6-ec21-49f4-b656-a07aed1f25c7-console-oauth-config\") pod \"console-84476cb65f-d4sm9\" (UID: \"72b366d6-ec21-49f4-b656-a07aed1f25c7\") " pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:03.693748 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.693690    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/72b366d6-ec21-49f4-b656-a07aed1f25c7-console-oauth-config\") pod \"console-84476cb65f-d4sm9\" (UID: \"72b366d6-ec21-49f4-b656-a07aed1f25c7\") " pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:03.693928 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.693760    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72b366d6-ec21-49f4-b656-a07aed1f25c7-trusted-ca-bundle\") pod \"console-84476cb65f-d4sm9\" (UID: \"72b366d6-ec21-49f4-b656-a07aed1f25c7\") " pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:03.693928 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.693799    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/72b366d6-ec21-49f4-b656-a07aed1f25c7-console-serving-cert\") pod \"console-84476cb65f-d4sm9\" (UID: \"72b366d6-ec21-49f4-b656-a07aed1f25c7\") " pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:03.693928 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.693839    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/72b366d6-ec21-49f4-b656-a07aed1f25c7-oauth-serving-cert\") pod \"console-84476cb65f-d4sm9\" (UID: \"72b366d6-ec21-49f4-b656-a07aed1f25c7\") " pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:03.693928 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.693870    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/72b366d6-ec21-49f4-b656-a07aed1f25c7-service-ca\") pod \"console-84476cb65f-d4sm9\" (UID: \"72b366d6-ec21-49f4-b656-a07aed1f25c7\") " pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:03.693928 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.693893    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psjr4\" (UniqueName: \"kubernetes.io/projected/72b366d6-ec21-49f4-b656-a07aed1f25c7-kube-api-access-psjr4\") pod \"console-84476cb65f-d4sm9\" (UID: \"72b366d6-ec21-49f4-b656-a07aed1f25c7\") " pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:03.693928 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.693920    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/72b366d6-ec21-49f4-b656-a07aed1f25c7-console-config\") pod \"console-84476cb65f-d4sm9\" (UID: \"72b366d6-ec21-49f4-b656-a07aed1f25c7\") " pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:03.694687 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.694664    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/72b366d6-ec21-49f4-b656-a07aed1f25c7-console-config\") pod \"console-84476cb65f-d4sm9\" (UID: \"72b366d6-ec21-49f4-b656-a07aed1f25c7\") " pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:03.694777 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.694736    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72b366d6-ec21-49f4-b656-a07aed1f25c7-trusted-ca-bundle\") pod \"console-84476cb65f-d4sm9\" (UID: \"72b366d6-ec21-49f4-b656-a07aed1f25c7\") " pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:03.694777 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.694736    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/72b366d6-ec21-49f4-b656-a07aed1f25c7-service-ca\") pod \"console-84476cb65f-d4sm9\" (UID: \"72b366d6-ec21-49f4-b656-a07aed1f25c7\") " pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:03.694847 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.694827    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/72b366d6-ec21-49f4-b656-a07aed1f25c7-oauth-serving-cert\") pod \"console-84476cb65f-d4sm9\" (UID: \"72b366d6-ec21-49f4-b656-a07aed1f25c7\") " pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:03.696305 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.696282    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/72b366d6-ec21-49f4-b656-a07aed1f25c7-console-oauth-config\") pod \"console-84476cb65f-d4sm9\" (UID: \"72b366d6-ec21-49f4-b656-a07aed1f25c7\") " pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:03.696397 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.696380    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/72b366d6-ec21-49f4-b656-a07aed1f25c7-console-serving-cert\") pod \"console-84476cb65f-d4sm9\" (UID: \"72b366d6-ec21-49f4-b656-a07aed1f25c7\") " pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:03.702984 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.702964    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psjr4\" (UniqueName: \"kubernetes.io/projected/72b366d6-ec21-49f4-b656-a07aed1f25c7-kube-api-access-psjr4\") pod \"console-84476cb65f-d4sm9\" (UID: \"72b366d6-ec21-49f4-b656-a07aed1f25c7\") " pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:03.807759 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.807731    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:03.947752 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:03.947726    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84476cb65f-d4sm9"]
Jul 26 17:57:03.949318 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:57:03.949292    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72b366d6_ec21_49f4_b656_a07aed1f25c7.slice/crio-69d863746b2d9f6de596f504ad7606ba82b26761656f9b38f79dc5143a5b9f27 WatchSource:0}: Error finding container 69d863746b2d9f6de596f504ad7606ba82b26761656f9b38f79dc5143a5b9f27: Status 404 returned error can't find the container with id 69d863746b2d9f6de596f504ad7606ba82b26761656f9b38f79dc5143a5b9f27
Jul 26 17:57:04.092874 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:04.092787    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84476cb65f-d4sm9" event={"ID":"72b366d6-ec21-49f4-b656-a07aed1f25c7","Type":"ContainerStarted","Data":"a7198fc15e49b41282731bd7d271d2f62d95cdecb757bb47eef68e5c4ef547ad"}
Jul 26 17:57:04.092874 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:04.092834    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84476cb65f-d4sm9" event={"ID":"72b366d6-ec21-49f4-b656-a07aed1f25c7","Type":"ContainerStarted","Data":"69d863746b2d9f6de596f504ad7606ba82b26761656f9b38f79dc5143a5b9f27"}
Jul 26 17:57:04.111594 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:04.111542    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84476cb65f-d4sm9" podStartSLOduration=1.111527496 podStartE2EDuration="1.111527496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:57:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:57:04.109971426 +0000 UTC m=+480.762162284" watchObservedRunningTime="2026-07-26 17:57:04.111527496 +0000 UTC m=+480.763718353"
Jul 26 17:57:13.808255 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:13.808222    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:13.808728 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:13.808298    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:13.813160 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:13.813140    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:14.143216 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:14.143131    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84476cb65f-d4sm9"
Jul 26 17:57:14.208871 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:14.208836    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76d9746fc5-hkp5d"]
Jul 26 17:57:21.203089 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:21.203056    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-trj28"]
Jul 26 17:57:21.207488 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:21.207468    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trj28"
Jul 26 17:57:21.209574 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:21.209546    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 26 17:57:21.209753 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:21.209736    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 26 17:57:21.209794 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:21.209764    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9sdhc\""
Jul 26 17:57:21.210378 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:21.210363    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 26 17:57:21.210445 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:21.210403    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 26 17:57:21.219414 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:21.219393    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-trj28"]
Jul 26 17:57:21.361201 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:21.361167    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b9c00b5-4b2c-4728-adb5-fe8a23d68d0d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-trj28\" (UID: \"6b9c00b5-4b2c-4728-adb5-fe8a23d68d0d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trj28"
Jul 26 17:57:21.361382 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:21.361209    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6b9c00b5-4b2c-4728-adb5-fe8a23d68d0d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-trj28\" (UID: \"6b9c00b5-4b2c-4728-adb5-fe8a23d68d0d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trj28"
Jul 26 17:57:21.361382 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:21.361351    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vswf\" (UniqueName: \"kubernetes.io/projected/6b9c00b5-4b2c-4728-adb5-fe8a23d68d0d-kube-api-access-7vswf\") pod \"kuadrant-console-plugin-66b5774985-trj28\" (UID: \"6b9c00b5-4b2c-4728-adb5-fe8a23d68d0d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trj28"
Jul 26 17:57:21.462836 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:21.462746    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vswf\" (UniqueName: \"kubernetes.io/projected/6b9c00b5-4b2c-4728-adb5-fe8a23d68d0d-kube-api-access-7vswf\") pod \"kuadrant-console-plugin-66b5774985-trj28\" (UID: \"6b9c00b5-4b2c-4728-adb5-fe8a23d68d0d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trj28"
Jul 26 17:57:21.462836 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:21.462794    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b9c00b5-4b2c-4728-adb5-fe8a23d68d0d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-trj28\" (UID: \"6b9c00b5-4b2c-4728-adb5-fe8a23d68d0d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trj28"
Jul 26 17:57:21.462836 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:21.462810    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6b9c00b5-4b2c-4728-adb5-fe8a23d68d0d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-trj28\" (UID: \"6b9c00b5-4b2c-4728-adb5-fe8a23d68d0d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trj28"
Jul 26 17:57:21.463562 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:21.463541    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6b9c00b5-4b2c-4728-adb5-fe8a23d68d0d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-trj28\" (UID: \"6b9c00b5-4b2c-4728-adb5-fe8a23d68d0d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trj28"
Jul 26 17:57:21.465204 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:21.465162    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b9c00b5-4b2c-4728-adb5-fe8a23d68d0d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-trj28\" (UID: \"6b9c00b5-4b2c-4728-adb5-fe8a23d68d0d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trj28"
Jul 26 17:57:21.471825 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:21.471798    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vswf\" (UniqueName: \"kubernetes.io/projected/6b9c00b5-4b2c-4728-adb5-fe8a23d68d0d-kube-api-access-7vswf\") pod \"kuadrant-console-plugin-66b5774985-trj28\" (UID: \"6b9c00b5-4b2c-4728-adb5-fe8a23d68d0d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trj28"
Jul 26 17:57:21.517703 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:21.517669    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trj28"
Jul 26 17:57:21.646654 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:21.646625    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-trj28"]
Jul 26 17:57:21.648361 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:57:21.648330    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b9c00b5_4b2c_4728_adb5_fe8a23d68d0d.slice/crio-2b101c398fecf3fb296519fb7b987ffb74c271c80863085e2ca673853552e8b6 WatchSource:0}: Error finding container 2b101c398fecf3fb296519fb7b987ffb74c271c80863085e2ca673853552e8b6: Status 404 returned error can't find the container with id 2b101c398fecf3fb296519fb7b987ffb74c271c80863085e2ca673853552e8b6
Jul 26 17:57:22.179277 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:22.179244    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trj28" event={"ID":"6b9c00b5-4b2c-4728-adb5-fe8a23d68d0d","Type":"ContainerStarted","Data":"2b101c398fecf3fb296519fb7b987ffb74c271c80863085e2ca673853552e8b6"}
Jul 26 17:57:39.229205 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:39.229107    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-76d9746fc5-hkp5d" podUID="01f883ba-c6d8-4071-8fe9-8f732fdb562f" containerName="console" containerID="cri-o://665e661e2ecc74bd9a03150d1a3503c00f2dbafa1797c11a0505e175677ace9d" gracePeriod=15
Jul 26 17:57:42.891076 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:42.891050    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:57:42.944102 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:42.944073    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01f883ba-c6d8-4071-8fe9-8f732fdb562f-trusted-ca-bundle\") pod \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") "
Jul 26 17:57:42.944102 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:42.944104    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hx5qn\" (UniqueName: \"kubernetes.io/projected/01f883ba-c6d8-4071-8fe9-8f732fdb562f-kube-api-access-hx5qn\") pod \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") "
Jul 26 17:57:42.944355 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:42.944144    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01f883ba-c6d8-4071-8fe9-8f732fdb562f-service-ca\") pod \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") "
Jul 26 17:57:42.944355 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:42.944263    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01f883ba-c6d8-4071-8fe9-8f732fdb562f-console-oauth-config\") pod \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") "
Jul 26 17:57:42.944355 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:42.944350    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01f883ba-c6d8-4071-8fe9-8f732fdb562f-console-config\") pod \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") "
Jul 26 17:57:42.944524 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:42.944377    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01f883ba-c6d8-4071-8fe9-8f732fdb562f-console-serving-cert\") pod \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") "
Jul 26 17:57:42.944524 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:42.944419    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01f883ba-c6d8-4071-8fe9-8f732fdb562f-oauth-serving-cert\") pod \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\" (UID: \"01f883ba-c6d8-4071-8fe9-8f732fdb562f\") "
Jul 26 17:57:42.944524 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:42.944510    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01f883ba-c6d8-4071-8fe9-8f732fdb562f-service-ca" (OuterVolumeSpecName: "service-ca") pod "01f883ba-c6d8-4071-8fe9-8f732fdb562f" (UID: "01f883ba-c6d8-4071-8fe9-8f732fdb562f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:57:42.944671 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:42.944525    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01f883ba-c6d8-4071-8fe9-8f732fdb562f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "01f883ba-c6d8-4071-8fe9-8f732fdb562f" (UID: "01f883ba-c6d8-4071-8fe9-8f732fdb562f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:57:42.944809 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:42.944791    2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01f883ba-c6d8-4071-8fe9-8f732fdb562f-trusted-ca-bundle\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:57:42.944862 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:42.944795    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01f883ba-c6d8-4071-8fe9-8f732fdb562f-console-config" (OuterVolumeSpecName: "console-config") pod "01f883ba-c6d8-4071-8fe9-8f732fdb562f" (UID: "01f883ba-c6d8-4071-8fe9-8f732fdb562f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:57:42.944862 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:42.944815    2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01f883ba-c6d8-4071-8fe9-8f732fdb562f-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:57:42.944939 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:42.944862    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01f883ba-c6d8-4071-8fe9-8f732fdb562f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "01f883ba-c6d8-4071-8fe9-8f732fdb562f" (UID: "01f883ba-c6d8-4071-8fe9-8f732fdb562f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:57:42.946418 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:42.946395    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01f883ba-c6d8-4071-8fe9-8f732fdb562f-kube-api-access-hx5qn" (OuterVolumeSpecName: "kube-api-access-hx5qn") pod "01f883ba-c6d8-4071-8fe9-8f732fdb562f" (UID: "01f883ba-c6d8-4071-8fe9-8f732fdb562f"). InnerVolumeSpecName "kube-api-access-hx5qn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:57:42.946496 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:42.946428    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01f883ba-c6d8-4071-8fe9-8f732fdb562f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "01f883ba-c6d8-4071-8fe9-8f732fdb562f" (UID: "01f883ba-c6d8-4071-8fe9-8f732fdb562f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 17:57:42.946496 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:42.946467    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01f883ba-c6d8-4071-8fe9-8f732fdb562f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "01f883ba-c6d8-4071-8fe9-8f732fdb562f" (UID: "01f883ba-c6d8-4071-8fe9-8f732fdb562f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 17:57:43.045362 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:43.045327    2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01f883ba-c6d8-4071-8fe9-8f732fdb562f-console-oauth-config\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:57:43.045362 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:43.045355    2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01f883ba-c6d8-4071-8fe9-8f732fdb562f-console-config\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:57:43.045362 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:43.045364    2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01f883ba-c6d8-4071-8fe9-8f732fdb562f-console-serving-cert\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:57:43.045628 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:43.045372    2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01f883ba-c6d8-4071-8fe9-8f732fdb562f-oauth-serving-cert\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:57:43.045628 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:43.045381    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx5qn\" (UniqueName: \"kubernetes.io/projected/01f883ba-c6d8-4071-8fe9-8f732fdb562f-kube-api-access-hx5qn\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:57:43.302967 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:43.302881    2570 generic.go:356] "Generic (PLEG): container finished" podID="01f883ba-c6d8-4071-8fe9-8f732fdb562f" containerID="665e661e2ecc74bd9a03150d1a3503c00f2dbafa1797c11a0505e175677ace9d" exitCode=0
Jul 26 17:57:43.302967 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:43.302928    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76d9746fc5-hkp5d"
Jul 26 17:57:43.303213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:43.302968    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76d9746fc5-hkp5d" event={"ID":"01f883ba-c6d8-4071-8fe9-8f732fdb562f","Type":"ContainerDied","Data":"665e661e2ecc74bd9a03150d1a3503c00f2dbafa1797c11a0505e175677ace9d"}
Jul 26 17:57:43.303213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:43.303015    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76d9746fc5-hkp5d" event={"ID":"01f883ba-c6d8-4071-8fe9-8f732fdb562f","Type":"ContainerDied","Data":"c91ae0be424fb90d4be1721f97ef883d5374a9d67f03ec77a8ef60cb43b8b113"}
Jul 26 17:57:43.303213 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:43.303036    2570 scope.go:117] "RemoveContainer" containerID="665e661e2ecc74bd9a03150d1a3503c00f2dbafa1797c11a0505e175677ace9d"
Jul 26 17:57:43.304603 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:43.304583    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trj28" event={"ID":"6b9c00b5-4b2c-4728-adb5-fe8a23d68d0d","Type":"ContainerStarted","Data":"997ea78856f6c7172fa34986d9845477dc077aee884fd38b606290c470274b75"}
Jul 26 17:57:43.312489 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:43.312471    2570 scope.go:117] "RemoveContainer" containerID="665e661e2ecc74bd9a03150d1a3503c00f2dbafa1797c11a0505e175677ace9d"
Jul 26 17:57:43.312752 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:57:43.312734    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"665e661e2ecc74bd9a03150d1a3503c00f2dbafa1797c11a0505e175677ace9d\": container with ID starting with 665e661e2ecc74bd9a03150d1a3503c00f2dbafa1797c11a0505e175677ace9d not found: ID does not exist" containerID="665e661e2ecc74bd9a03150d1a3503c00f2dbafa1797c11a0505e175677ace9d"
Jul 26 17:57:43.312798 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:43.312761    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"665e661e2ecc74bd9a03150d1a3503c00f2dbafa1797c11a0505e175677ace9d"} err="failed to get container status \"665e661e2ecc74bd9a03150d1a3503c00f2dbafa1797c11a0505e175677ace9d\": rpc error: code = NotFound desc = could not find container \"665e661e2ecc74bd9a03150d1a3503c00f2dbafa1797c11a0505e175677ace9d\": container with ID starting with 665e661e2ecc74bd9a03150d1a3503c00f2dbafa1797c11a0505e175677ace9d not found: ID does not exist"
Jul 26 17:57:43.323259 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:43.323213    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-trj28" podStartSLOduration=1.170420915 podStartE2EDuration="22.32319592s" totalImagesPullingTime="21.152775005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:57:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:57:43.320887074 +0000 UTC m=+519.973077949" watchObservedRunningTime="2026-07-26 17:57:43.32319592 +0000 UTC m=+519.975386780"
Jul 26 17:57:43.336434 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:43.336408    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76d9746fc5-hkp5d"]
Jul 26 17:57:43.341827 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:43.341794    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76d9746fc5-hkp5d"]
Jul 26 17:57:43.839651 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:57:43.839615    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01f883ba-c6d8-4071-8fe9-8f732fdb562f" path="/var/lib/kubelet/pods/01f883ba-c6d8-4071-8fe9-8f732fdb562f/volumes"
Jul 26 17:58:15.157519 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:15.157483    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z2prd"]
Jul 26 17:58:15.157986 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:15.157813    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01f883ba-c6d8-4071-8fe9-8f732fdb562f" containerName="console"
Jul 26 17:58:15.157986 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:15.157824    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="01f883ba-c6d8-4071-8fe9-8f732fdb562f" containerName="console"
Jul 26 17:58:15.157986 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:15.157903    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="01f883ba-c6d8-4071-8fe9-8f732fdb562f" containerName="console"
Jul 26 17:58:15.188080 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:15.188048    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z2prd"]
Jul 26 17:58:15.188245 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:15.188169    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z2prd"
Jul 26 17:58:15.190414 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:15.190390    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 26 17:58:15.248676 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:15.248644    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z2prd"]
Jul 26 17:58:15.314169 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:15.314132    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgxdh\" (UniqueName: \"kubernetes.io/projected/c3304799-cceb-4325-91db-cd9526e84173-kube-api-access-kgxdh\") pod \"limitador-limitador-64c94fd64f-z2prd\" (UID: \"c3304799-cceb-4325-91db-cd9526e84173\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z2prd"
Jul 26 17:58:15.314169 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:15.314171    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c3304799-cceb-4325-91db-cd9526e84173-config-file\") pod \"limitador-limitador-64c94fd64f-z2prd\" (UID: \"c3304799-cceb-4325-91db-cd9526e84173\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z2prd"
Jul 26 17:58:15.414921 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:15.414833    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgxdh\" (UniqueName: \"kubernetes.io/projected/c3304799-cceb-4325-91db-cd9526e84173-kube-api-access-kgxdh\") pod \"limitador-limitador-64c94fd64f-z2prd\" (UID: \"c3304799-cceb-4325-91db-cd9526e84173\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z2prd"
Jul 26 17:58:15.414921 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:15.414876    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c3304799-cceb-4325-91db-cd9526e84173-config-file\") pod \"limitador-limitador-64c94fd64f-z2prd\" (UID: \"c3304799-cceb-4325-91db-cd9526e84173\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z2prd"
Jul 26 17:58:15.415571 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:15.415552    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c3304799-cceb-4325-91db-cd9526e84173-config-file\") pod \"limitador-limitador-64c94fd64f-z2prd\" (UID: \"c3304799-cceb-4325-91db-cd9526e84173\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z2prd"
Jul 26 17:58:15.422699 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:15.422676    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgxdh\" (UniqueName: \"kubernetes.io/projected/c3304799-cceb-4325-91db-cd9526e84173-kube-api-access-kgxdh\") pod \"limitador-limitador-64c94fd64f-z2prd\" (UID: \"c3304799-cceb-4325-91db-cd9526e84173\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z2prd"
Jul 26 17:58:15.498713 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:15.498675    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z2prd"
Jul 26 17:58:15.646688 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:15.646654    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z2prd"]
Jul 26 17:58:15.648932 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:58:15.648904    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3304799_cceb_4325_91db_cd9526e84173.slice/crio-93b15a928ef055ba537d6aec0c2a16dbaef86efab0b1513e8b9e28d0bdc58a56 WatchSource:0}: Error finding container 93b15a928ef055ba537d6aec0c2a16dbaef86efab0b1513e8b9e28d0bdc58a56: Status 404 returned error can't find the container with id 93b15a928ef055ba537d6aec0c2a16dbaef86efab0b1513e8b9e28d0bdc58a56
Jul 26 17:58:15.959910 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:15.959878    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-7bwpv"]
Jul 26 17:58:15.964466 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:15.964445    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7bwpv"
Jul 26 17:58:15.966396 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:15.966374    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-vv6rg\""
Jul 26 17:58:15.970138 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:15.970103    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-7bwpv"]
Jul 26 17:58:16.121970 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:16.121931    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpw9s\" (UniqueName: \"kubernetes.io/projected/78b8cd81-d1d8-4dfe-8d34-960466335ac6-kube-api-access-cpw9s\") pod \"authorino-58cd955675-7bwpv\" (UID: \"78b8cd81-d1d8-4dfe-8d34-960466335ac6\") " pod="kuadrant-system/authorino-58cd955675-7bwpv"
Jul 26 17:58:16.159102 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:16.159065    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-4cjbv"]
Jul 26 17:58:16.162455 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:16.162436    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4cjbv"
Jul 26 17:58:16.169989 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:16.169965    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4cjbv"]
Jul 26 17:58:16.223345 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:16.223274    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpw9s\" (UniqueName: \"kubernetes.io/projected/78b8cd81-d1d8-4dfe-8d34-960466335ac6-kube-api-access-cpw9s\") pod \"authorino-58cd955675-7bwpv\" (UID: \"78b8cd81-d1d8-4dfe-8d34-960466335ac6\") " pod="kuadrant-system/authorino-58cd955675-7bwpv"
Jul 26 17:58:16.231029 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:16.231000    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpw9s\" (UniqueName: \"kubernetes.io/projected/78b8cd81-d1d8-4dfe-8d34-960466335ac6-kube-api-access-cpw9s\") pod \"authorino-58cd955675-7bwpv\" (UID: \"78b8cd81-d1d8-4dfe-8d34-960466335ac6\") " pod="kuadrant-system/authorino-58cd955675-7bwpv"
Jul 26 17:58:16.274927 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:16.274890    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7bwpv"
Jul 26 17:58:16.324194 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:16.324137    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl4th\" (UniqueName: \"kubernetes.io/projected/6930710a-98d4-4622-b0f1-a724f5996bc0-kube-api-access-hl4th\") pod \"authorino-75d556bd77-4cjbv\" (UID: \"6930710a-98d4-4622-b0f1-a724f5996bc0\") " pod="kuadrant-system/authorino-75d556bd77-4cjbv"
Jul 26 17:58:16.425301 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:16.425255    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl4th\" (UniqueName: \"kubernetes.io/projected/6930710a-98d4-4622-b0f1-a724f5996bc0-kube-api-access-hl4th\") pod \"authorino-75d556bd77-4cjbv\" (UID: \"6930710a-98d4-4622-b0f1-a724f5996bc0\") " pod="kuadrant-system/authorino-75d556bd77-4cjbv"
Jul 26 17:58:16.437669 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:16.437610    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl4th\" (UniqueName: \"kubernetes.io/projected/6930710a-98d4-4622-b0f1-a724f5996bc0-kube-api-access-hl4th\") pod \"authorino-75d556bd77-4cjbv\" (UID: \"6930710a-98d4-4622-b0f1-a724f5996bc0\") " pod="kuadrant-system/authorino-75d556bd77-4cjbv"
Jul 26 17:58:16.439952 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:16.439895    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-7bwpv"]
Jul 26 17:58:16.446255 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:58:16.446221    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod78b8cd81_d1d8_4dfe_8d34_960466335ac6.slice/crio-1f31686cbb86837571ccd49de359a9ec0fd2cc03bab2618f67d9e49ca8fffc7a WatchSource:0}: Error finding container 1f31686cbb86837571ccd49de359a9ec0fd2cc03bab2618f67d9e49ca8fffc7a: Status 404 returned error can't find the container with id 1f31686cbb86837571ccd49de359a9ec0fd2cc03bab2618f67d9e49ca8fffc7a
Jul 26 17:58:16.471984 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:16.471955    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7bwpv" event={"ID":"78b8cd81-d1d8-4dfe-8d34-960466335ac6","Type":"ContainerStarted","Data":"1f31686cbb86837571ccd49de359a9ec0fd2cc03bab2618f67d9e49ca8fffc7a"}
Jul 26 17:58:16.472378 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:16.472359    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4cjbv"
Jul 26 17:58:16.475227 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:16.475064    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z2prd" event={"ID":"c3304799-cceb-4325-91db-cd9526e84173","Type":"ContainerStarted","Data":"93b15a928ef055ba537d6aec0c2a16dbaef86efab0b1513e8b9e28d0bdc58a56"}
Jul 26 17:58:16.650012 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:16.649978    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4cjbv"]
Jul 26 17:58:16.651988 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:58:16.651953    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6930710a_98d4_4622_b0f1_a724f5996bc0.slice/crio-c20d088ffcaf73230756eda5b3a2e44ae3850c1282a938336c4a1f2e0426a623 WatchSource:0}: Error finding container c20d088ffcaf73230756eda5b3a2e44ae3850c1282a938336c4a1f2e0426a623: Status 404 returned error can't find the container with id c20d088ffcaf73230756eda5b3a2e44ae3850c1282a938336c4a1f2e0426a623
Jul 26 17:58:17.488962 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:17.488884    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4cjbv" event={"ID":"6930710a-98d4-4622-b0f1-a724f5996bc0","Type":"ContainerStarted","Data":"c20d088ffcaf73230756eda5b3a2e44ae3850c1282a938336c4a1f2e0426a623"}
Jul 26 17:58:21.514725 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:21.514690    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7bwpv" event={"ID":"78b8cd81-d1d8-4dfe-8d34-960466335ac6","Type":"ContainerStarted","Data":"d59b7e925b180c9517fd1e3222c55d76f633020b0009bf89d6ed3813dc499777"}
Jul 26 17:58:21.516239 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:21.516208    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z2prd" event={"ID":"c3304799-cceb-4325-91db-cd9526e84173","Type":"ContainerStarted","Data":"5dad50ba8ab2909c818977cd268ccd26d05992cd369339098f406777e52d8588"}
Jul 26 17:58:21.516357 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:21.516302    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-z2prd"
Jul 26 17:58:21.517620 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:21.517598    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4cjbv" event={"ID":"6930710a-98d4-4622-b0f1-a724f5996bc0","Type":"ContainerStarted","Data":"93066599034f4b087b957e61bbf7e81fa6522eeeb64a30b0124394b0ac8f4a99"}
Jul 26 17:58:21.529731 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:21.529682    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-7bwpv" podStartSLOduration=2.165826775 podStartE2EDuration="6.529667746s" totalImagesPullingTime="4.363840971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:58:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:58:21.52786734 +0000 UTC m=+558.180058200" watchObservedRunningTime="2026-07-26 17:58:21.529667746 +0000 UTC m=+558.181858603"
Jul 26 17:58:21.544806 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:21.544744    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-4cjbv" podStartSLOduration=1.395877564 podStartE2EDuration="5.544723679s" totalImagesPullingTime="4.148846115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:58:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:58:21.541426095 +0000 UTC m=+558.193616956" watchObservedRunningTime="2026-07-26 17:58:21.544723679 +0000 UTC m=+558.196914541"
Jul 26 17:58:21.559212 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:21.559162    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-z2prd" podStartSLOduration=1.407934867 podStartE2EDuration="6.559146342s" totalImagesPullingTime="5.151211475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:58:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:58:21.556664698 +0000 UTC m=+558.208855557" watchObservedRunningTime="2026-07-26 17:58:21.559146342 +0000 UTC m=+558.211337201"
Jul 26 17:58:21.572481 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:21.572449    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-7bwpv"]
Jul 26 17:58:23.528324 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:23.528283    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-7bwpv" podUID="78b8cd81-d1d8-4dfe-8d34-960466335ac6" containerName="authorino" containerID="cri-o://d59b7e925b180c9517fd1e3222c55d76f633020b0009bf89d6ed3813dc499777" gracePeriod=30
Jul 26 17:58:23.778421 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:23.778362    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7bwpv"
Jul 26 17:58:23.895951 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:23.895918    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpw9s\" (UniqueName: \"kubernetes.io/projected/78b8cd81-d1d8-4dfe-8d34-960466335ac6-kube-api-access-cpw9s\") pod \"78b8cd81-d1d8-4dfe-8d34-960466335ac6\" (UID: \"78b8cd81-d1d8-4dfe-8d34-960466335ac6\") "
Jul 26 17:58:23.898038 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:23.898006    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78b8cd81-d1d8-4dfe-8d34-960466335ac6-kube-api-access-cpw9s" (OuterVolumeSpecName: "kube-api-access-cpw9s") pod "78b8cd81-d1d8-4dfe-8d34-960466335ac6" (UID: "78b8cd81-d1d8-4dfe-8d34-960466335ac6"). InnerVolumeSpecName "kube-api-access-cpw9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:58:23.996973 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:23.996935    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpw9s\" (UniqueName: \"kubernetes.io/projected/78b8cd81-d1d8-4dfe-8d34-960466335ac6-kube-api-access-cpw9s\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:58:24.534255 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:24.534217    2570 generic.go:356] "Generic (PLEG): container finished" podID="78b8cd81-d1d8-4dfe-8d34-960466335ac6" containerID="d59b7e925b180c9517fd1e3222c55d76f633020b0009bf89d6ed3813dc499777" exitCode=0
Jul 26 17:58:24.534662 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:24.534260    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7bwpv"
Jul 26 17:58:24.534662 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:24.534301    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7bwpv" event={"ID":"78b8cd81-d1d8-4dfe-8d34-960466335ac6","Type":"ContainerDied","Data":"d59b7e925b180c9517fd1e3222c55d76f633020b0009bf89d6ed3813dc499777"}
Jul 26 17:58:24.534662 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:24.534345    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7bwpv" event={"ID":"78b8cd81-d1d8-4dfe-8d34-960466335ac6","Type":"ContainerDied","Data":"1f31686cbb86837571ccd49de359a9ec0fd2cc03bab2618f67d9e49ca8fffc7a"}
Jul 26 17:58:24.534662 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:24.534365    2570 scope.go:117] "RemoveContainer" containerID="d59b7e925b180c9517fd1e3222c55d76f633020b0009bf89d6ed3813dc499777"
Jul 26 17:58:24.543192 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:24.543172    2570 scope.go:117] "RemoveContainer" containerID="d59b7e925b180c9517fd1e3222c55d76f633020b0009bf89d6ed3813dc499777"
Jul 26 17:58:24.543484 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:58:24.543468    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d59b7e925b180c9517fd1e3222c55d76f633020b0009bf89d6ed3813dc499777\": container with ID starting with d59b7e925b180c9517fd1e3222c55d76f633020b0009bf89d6ed3813dc499777 not found: ID does not exist" containerID="d59b7e925b180c9517fd1e3222c55d76f633020b0009bf89d6ed3813dc499777"
Jul 26 17:58:24.543541 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:24.543492    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d59b7e925b180c9517fd1e3222c55d76f633020b0009bf89d6ed3813dc499777"} err="failed to get container status \"d59b7e925b180c9517fd1e3222c55d76f633020b0009bf89d6ed3813dc499777\": rpc error: code = NotFound desc = could not find container \"d59b7e925b180c9517fd1e3222c55d76f633020b0009bf89d6ed3813dc499777\": container with ID starting with d59b7e925b180c9517fd1e3222c55d76f633020b0009bf89d6ed3813dc499777 not found: ID does not exist"
Jul 26 17:58:24.555249 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:24.555226    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-7bwpv"]
Jul 26 17:58:24.557135 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:24.557099    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-7bwpv"]
Jul 26 17:58:25.838300 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:25.838268    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78b8cd81-d1d8-4dfe-8d34-960466335ac6" path="/var/lib/kubelet/pods/78b8cd81-d1d8-4dfe-8d34-960466335ac6/volumes"
Jul 26 17:58:29.922591 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:29.922550    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z2prd"]
Jul 26 17:58:29.923066 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:29.922771    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-z2prd" podUID="c3304799-cceb-4325-91db-cd9526e84173" containerName="limitador" containerID="cri-o://5dad50ba8ab2909c818977cd268ccd26d05992cd369339098f406777e52d8588" gracePeriod=30
Jul 26 17:58:29.923468 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:29.923450    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-z2prd"
Jul 26 17:58:30.481848 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:30.481793    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z2prd"
Jul 26 17:58:30.547171 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:30.547139    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c3304799-cceb-4325-91db-cd9526e84173-config-file\") pod \"c3304799-cceb-4325-91db-cd9526e84173\" (UID: \"c3304799-cceb-4325-91db-cd9526e84173\") "
Jul 26 17:58:30.547171 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:30.547173    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgxdh\" (UniqueName: \"kubernetes.io/projected/c3304799-cceb-4325-91db-cd9526e84173-kube-api-access-kgxdh\") pod \"c3304799-cceb-4325-91db-cd9526e84173\" (UID: \"c3304799-cceb-4325-91db-cd9526e84173\") "
Jul 26 17:58:30.547475 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:30.547452    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3304799-cceb-4325-91db-cd9526e84173-config-file" (OuterVolumeSpecName: "config-file") pod "c3304799-cceb-4325-91db-cd9526e84173" (UID: "c3304799-cceb-4325-91db-cd9526e84173"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 17:58:30.549313 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:30.549293    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3304799-cceb-4325-91db-cd9526e84173-kube-api-access-kgxdh" (OuterVolumeSpecName: "kube-api-access-kgxdh") pod "c3304799-cceb-4325-91db-cd9526e84173" (UID: "c3304799-cceb-4325-91db-cd9526e84173"). InnerVolumeSpecName "kube-api-access-kgxdh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:58:30.565946 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:30.565920    2570 generic.go:356] "Generic (PLEG): container finished" podID="c3304799-cceb-4325-91db-cd9526e84173" containerID="5dad50ba8ab2909c818977cd268ccd26d05992cd369339098f406777e52d8588" exitCode=0
Jul 26 17:58:30.566047 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:30.565964    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z2prd"
Jul 26 17:58:30.566047 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:30.566001    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z2prd" event={"ID":"c3304799-cceb-4325-91db-cd9526e84173","Type":"ContainerDied","Data":"5dad50ba8ab2909c818977cd268ccd26d05992cd369339098f406777e52d8588"}
Jul 26 17:58:30.566147 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:30.566059    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z2prd" event={"ID":"c3304799-cceb-4325-91db-cd9526e84173","Type":"ContainerDied","Data":"93b15a928ef055ba537d6aec0c2a16dbaef86efab0b1513e8b9e28d0bdc58a56"}
Jul 26 17:58:30.566147 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:30.566082    2570 scope.go:117] "RemoveContainer" containerID="5dad50ba8ab2909c818977cd268ccd26d05992cd369339098f406777e52d8588"
Jul 26 17:58:30.575479 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:30.575460    2570 scope.go:117] "RemoveContainer" containerID="5dad50ba8ab2909c818977cd268ccd26d05992cd369339098f406777e52d8588"
Jul 26 17:58:30.575765 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:58:30.575746    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dad50ba8ab2909c818977cd268ccd26d05992cd369339098f406777e52d8588\": container with ID starting with 5dad50ba8ab2909c818977cd268ccd26d05992cd369339098f406777e52d8588 not found: ID does not exist" containerID="5dad50ba8ab2909c818977cd268ccd26d05992cd369339098f406777e52d8588"
Jul 26 17:58:30.575806 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:30.575773    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dad50ba8ab2909c818977cd268ccd26d05992cd369339098f406777e52d8588"} err="failed to get container status \"5dad50ba8ab2909c818977cd268ccd26d05992cd369339098f406777e52d8588\": rpc error: code = NotFound desc = could not find container \"5dad50ba8ab2909c818977cd268ccd26d05992cd369339098f406777e52d8588\": container with ID starting with 5dad50ba8ab2909c818977cd268ccd26d05992cd369339098f406777e52d8588 not found: ID does not exist"
Jul 26 17:58:30.587078 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:30.587056    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z2prd"]
Jul 26 17:58:30.589217 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:30.589197    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z2prd"]
Jul 26 17:58:30.648525 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:30.648502    2570 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c3304799-cceb-4325-91db-cd9526e84173-config-file\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:58:30.648525 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:30.648526    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgxdh\" (UniqueName: \"kubernetes.io/projected/c3304799-cceb-4325-91db-cd9526e84173-kube-api-access-kgxdh\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:58:31.837217 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:31.837179    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3304799-cceb-4325-91db-cd9526e84173" path="/var/lib/kubelet/pods/c3304799-cceb-4325-91db-cd9526e84173/volumes"
Jul 26 17:58:47.246898 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.246862    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pt4bf"]
Jul 26 17:58:47.247370 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.247230    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78b8cd81-d1d8-4dfe-8d34-960466335ac6" containerName="authorino"
Jul 26 17:58:47.247370 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.247242    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="78b8cd81-d1d8-4dfe-8d34-960466335ac6" containerName="authorino"
Jul 26 17:58:47.247370 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.247259    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3304799-cceb-4325-91db-cd9526e84173" containerName="limitador"
Jul 26 17:58:47.247370 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.247264    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3304799-cceb-4325-91db-cd9526e84173" containerName="limitador"
Jul 26 17:58:47.247370 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.247332    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3304799-cceb-4325-91db-cd9526e84173" containerName="limitador"
Jul 26 17:58:47.247370 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.247341    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="78b8cd81-d1d8-4dfe-8d34-960466335ac6" containerName="authorino"
Jul 26 17:58:47.251745 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.251727    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-pt4bf"
Jul 26 17:58:47.256579 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.256551    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pt4bf"]
Jul 26 17:58:47.280677 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.280652    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsfv7\" (UniqueName: \"kubernetes.io/projected/191b19b3-58ad-48c5-97de-08d3a61feb5b-kube-api-access-zsfv7\") pod \"authorino-5d4b67c446-pt4bf\" (UID: \"191b19b3-58ad-48c5-97de-08d3a61feb5b\") " pod="kuadrant-system/authorino-5d4b67c446-pt4bf"
Jul 26 17:58:47.381297 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.381263    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsfv7\" (UniqueName: \"kubernetes.io/projected/191b19b3-58ad-48c5-97de-08d3a61feb5b-kube-api-access-zsfv7\") pod \"authorino-5d4b67c446-pt4bf\" (UID: \"191b19b3-58ad-48c5-97de-08d3a61feb5b\") " pod="kuadrant-system/authorino-5d4b67c446-pt4bf"
Jul 26 17:58:47.389494 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.389464    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsfv7\" (UniqueName: \"kubernetes.io/projected/191b19b3-58ad-48c5-97de-08d3a61feb5b-kube-api-access-zsfv7\") pod \"authorino-5d4b67c446-pt4bf\" (UID: \"191b19b3-58ad-48c5-97de-08d3a61feb5b\") " pod="kuadrant-system/authorino-5d4b67c446-pt4bf"
Jul 26 17:58:47.475287 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.475255    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pt4bf"]
Jul 26 17:58:47.475510 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.475499    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-pt4bf"
Jul 26 17:58:47.499607 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.499544    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-944856bf4-k5dzc"]
Jul 26 17:58:47.503933 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.503913    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-944856bf4-k5dzc"
Jul 26 17:58:47.512420 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.512396    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-944856bf4-k5dzc"]
Jul 26 17:58:47.582450 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.582415    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mswwd\" (UniqueName: \"kubernetes.io/projected/6e4d3bfd-0159-40c6-89ce-16d411a1574c-kube-api-access-mswwd\") pod \"authorino-944856bf4-k5dzc\" (UID: \"6e4d3bfd-0159-40c6-89ce-16d411a1574c\") " pod="kuadrant-system/authorino-944856bf4-k5dzc"
Jul 26 17:58:47.608269 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.608242    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pt4bf"]
Jul 26 17:58:47.610611 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:58:47.610584    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod191b19b3_58ad_48c5_97de_08d3a61feb5b.slice/crio-017464c5b9b1d17c6471c612e6edba38ec167fe07d3b0861e932c4851826ba9c WatchSource:0}: Error finding container 017464c5b9b1d17c6471c612e6edba38ec167fe07d3b0861e932c4851826ba9c: Status 404 returned error can't find the container with id 017464c5b9b1d17c6471c612e6edba38ec167fe07d3b0861e932c4851826ba9c
Jul 26 17:58:47.619438 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.619409    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-944856bf4-k5dzc"]
Jul 26 17:58:47.619677 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:58:47.619656    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-mswwd], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-944856bf4-k5dzc" podUID="6e4d3bfd-0159-40c6-89ce-16d411a1574c"
Jul 26 17:58:47.646779 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.646742    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7b758bfb7d-fdlfz"]
Jul 26 17:58:47.650330 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.650310    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b758bfb7d-fdlfz"
Jul 26 17:58:47.652443 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.652422    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 26 17:58:47.657085 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.657064    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b758bfb7d-fdlfz"]
Jul 26 17:58:47.661971 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.661940    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-pt4bf" event={"ID":"191b19b3-58ad-48c5-97de-08d3a61feb5b","Type":"ContainerStarted","Data":"017464c5b9b1d17c6471c612e6edba38ec167fe07d3b0861e932c4851826ba9c"}
Jul 26 17:58:47.662082 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.661951    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-944856bf4-k5dzc"
Jul 26 17:58:47.668805 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.668785    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-944856bf4-k5dzc"
Jul 26 17:58:47.683367 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.683348    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mswwd\" (UniqueName: \"kubernetes.io/projected/6e4d3bfd-0159-40c6-89ce-16d411a1574c-kube-api-access-mswwd\") pod \"authorino-944856bf4-k5dzc\" (UID: \"6e4d3bfd-0159-40c6-89ce-16d411a1574c\") " pod="kuadrant-system/authorino-944856bf4-k5dzc"
Jul 26 17:58:47.683484 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.683406    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1f1c1680-70be-439c-a8b6-5d63fffd6704-tls-cert\") pod \"authorino-7b758bfb7d-fdlfz\" (UID: \"1f1c1680-70be-439c-a8b6-5d63fffd6704\") " pod="kuadrant-system/authorino-7b758bfb7d-fdlfz"
Jul 26 17:58:47.683484 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.683424    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwqqm\" (UniqueName: \"kubernetes.io/projected/1f1c1680-70be-439c-a8b6-5d63fffd6704-kube-api-access-kwqqm\") pod \"authorino-7b758bfb7d-fdlfz\" (UID: \"1f1c1680-70be-439c-a8b6-5d63fffd6704\") " pod="kuadrant-system/authorino-7b758bfb7d-fdlfz"
Jul 26 17:58:47.696182 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.696161    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mswwd\" (UniqueName: \"kubernetes.io/projected/6e4d3bfd-0159-40c6-89ce-16d411a1574c-kube-api-access-mswwd\") pod \"authorino-944856bf4-k5dzc\" (UID: \"6e4d3bfd-0159-40c6-89ce-16d411a1574c\") " pod="kuadrant-system/authorino-944856bf4-k5dzc"
Jul 26 17:58:47.784679 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.784651    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mswwd\" (UniqueName: \"kubernetes.io/projected/6e4d3bfd-0159-40c6-89ce-16d411a1574c-kube-api-access-mswwd\") pod \"6e4d3bfd-0159-40c6-89ce-16d411a1574c\" (UID: \"6e4d3bfd-0159-40c6-89ce-16d411a1574c\") "
Jul 26 17:58:47.784844 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.784811    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1f1c1680-70be-439c-a8b6-5d63fffd6704-tls-cert\") pod \"authorino-7b758bfb7d-fdlfz\" (UID: \"1f1c1680-70be-439c-a8b6-5d63fffd6704\") " pod="kuadrant-system/authorino-7b758bfb7d-fdlfz"
Jul 26 17:58:47.784844 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.784831    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwqqm\" (UniqueName: \"kubernetes.io/projected/1f1c1680-70be-439c-a8b6-5d63fffd6704-kube-api-access-kwqqm\") pod \"authorino-7b758bfb7d-fdlfz\" (UID: \"1f1c1680-70be-439c-a8b6-5d63fffd6704\") " pod="kuadrant-system/authorino-7b758bfb7d-fdlfz"
Jul 26 17:58:47.786768 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.786739    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e4d3bfd-0159-40c6-89ce-16d411a1574c-kube-api-access-mswwd" (OuterVolumeSpecName: "kube-api-access-mswwd") pod "6e4d3bfd-0159-40c6-89ce-16d411a1574c" (UID: "6e4d3bfd-0159-40c6-89ce-16d411a1574c"). InnerVolumeSpecName "kube-api-access-mswwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:58:47.787139 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.787102    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1f1c1680-70be-439c-a8b6-5d63fffd6704-tls-cert\") pod \"authorino-7b758bfb7d-fdlfz\" (UID: \"1f1c1680-70be-439c-a8b6-5d63fffd6704\") " pod="kuadrant-system/authorino-7b758bfb7d-fdlfz"
Jul 26 17:58:47.791986 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.791966    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwqqm\" (UniqueName: \"kubernetes.io/projected/1f1c1680-70be-439c-a8b6-5d63fffd6704-kube-api-access-kwqqm\") pod \"authorino-7b758bfb7d-fdlfz\" (UID: \"1f1c1680-70be-439c-a8b6-5d63fffd6704\") " pod="kuadrant-system/authorino-7b758bfb7d-fdlfz"
Jul 26 17:58:47.886036 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.886009    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mswwd\" (UniqueName: \"kubernetes.io/projected/6e4d3bfd-0159-40c6-89ce-16d411a1574c-kube-api-access-mswwd\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:58:47.960920 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:47.960878    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b758bfb7d-fdlfz"
Jul 26 17:58:48.107861 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:48.107822    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b758bfb7d-fdlfz"]
Jul 26 17:58:48.110161 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:58:48.110135    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f1c1680_70be_439c_a8b6_5d63fffd6704.slice/crio-13dc8c009af2d7f8ac58a76da301ab4ebdc23554b2aa43410c582db392562bf8 WatchSource:0}: Error finding container 13dc8c009af2d7f8ac58a76da301ab4ebdc23554b2aa43410c582db392562bf8: Status 404 returned error can't find the container with id 13dc8c009af2d7f8ac58a76da301ab4ebdc23554b2aa43410c582db392562bf8
Jul 26 17:58:48.668777 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:48.668685    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-pt4bf" event={"ID":"191b19b3-58ad-48c5-97de-08d3a61feb5b","Type":"ContainerStarted","Data":"4b63e08b9a72d64ceacb71c07481caf09b8d668e8deca10121c830560bed0e44"}
Jul 26 17:58:48.668777 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:48.668706    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-pt4bf" podUID="191b19b3-58ad-48c5-97de-08d3a61feb5b" containerName="authorino" containerID="cri-o://4b63e08b9a72d64ceacb71c07481caf09b8d668e8deca10121c830560bed0e44" gracePeriod=30
Jul 26 17:58:48.670265 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:48.670242    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b758bfb7d-fdlfz" event={"ID":"1f1c1680-70be-439c-a8b6-5d63fffd6704","Type":"ContainerStarted","Data":"d6339c07341812cbef97ed92eebf5ecd6c02924dff8175d63939a196f1e252f2"}
Jul 26 17:58:48.670359 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:48.670266    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-944856bf4-k5dzc"
Jul 26 17:58:48.670359 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:48.670282    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b758bfb7d-fdlfz" event={"ID":"1f1c1680-70be-439c-a8b6-5d63fffd6704","Type":"ContainerStarted","Data":"13dc8c009af2d7f8ac58a76da301ab4ebdc23554b2aa43410c582db392562bf8"}
Jul 26 17:58:48.683315 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:48.683270    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-pt4bf" podStartSLOduration=1.316417476 podStartE2EDuration="1.683257553s" totalImagesPullingTime="366.840077ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:58:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:58:48.682699174 +0000 UTC m=+585.334890036" watchObservedRunningTime="2026-07-26 17:58:48.683257553 +0000 UTC m=+585.335448472"
Jul 26 17:58:48.703424 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:48.703401    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-944856bf4-k5dzc"]
Jul 26 17:58:48.707096 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:48.707074    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-944856bf4-k5dzc"]
Jul 26 17:58:48.719980 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:48.719933    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7b758bfb7d-fdlfz" podStartSLOduration=1.443858286 podStartE2EDuration="1.719918617s" totalImagesPullingTime="276.060331ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:58:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:58:48.719381321 +0000 UTC m=+585.371572182" watchObservedRunningTime="2026-07-26 17:58:48.719918617 +0000 UTC m=+585.372109475"
Jul 26 17:58:48.747707 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:48.747677    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4cjbv"]
Jul 26 17:58:48.747866 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:48.747846    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-4cjbv" podUID="6930710a-98d4-4622-b0f1-a724f5996bc0" containerName="authorino" containerID="cri-o://93066599034f4b087b957e61bbf7e81fa6522eeeb64a30b0124394b0ac8f4a99" gracePeriod=30
Jul 26 17:58:48.996540 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:48.996516    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-pt4bf"
Jul 26 17:58:49.001996 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.001972    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4cjbv"
Jul 26 17:58:49.096271 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.096238    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsfv7\" (UniqueName: \"kubernetes.io/projected/191b19b3-58ad-48c5-97de-08d3a61feb5b-kube-api-access-zsfv7\") pod \"191b19b3-58ad-48c5-97de-08d3a61feb5b\" (UID: \"191b19b3-58ad-48c5-97de-08d3a61feb5b\") "
Jul 26 17:58:49.096485 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.096354    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hl4th\" (UniqueName: \"kubernetes.io/projected/6930710a-98d4-4622-b0f1-a724f5996bc0-kube-api-access-hl4th\") pod \"6930710a-98d4-4622-b0f1-a724f5996bc0\" (UID: \"6930710a-98d4-4622-b0f1-a724f5996bc0\") "
Jul 26 17:58:49.098451 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.098413    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/191b19b3-58ad-48c5-97de-08d3a61feb5b-kube-api-access-zsfv7" (OuterVolumeSpecName: "kube-api-access-zsfv7") pod "191b19b3-58ad-48c5-97de-08d3a61feb5b" (UID: "191b19b3-58ad-48c5-97de-08d3a61feb5b"). InnerVolumeSpecName "kube-api-access-zsfv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:58:49.098451 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.098441    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6930710a-98d4-4622-b0f1-a724f5996bc0-kube-api-access-hl4th" (OuterVolumeSpecName: "kube-api-access-hl4th") pod "6930710a-98d4-4622-b0f1-a724f5996bc0" (UID: "6930710a-98d4-4622-b0f1-a724f5996bc0"). InnerVolumeSpecName "kube-api-access-hl4th". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 17:58:49.197095 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.197010    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsfv7\" (UniqueName: \"kubernetes.io/projected/191b19b3-58ad-48c5-97de-08d3a61feb5b-kube-api-access-zsfv7\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:58:49.197095 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.197040    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hl4th\" (UniqueName: \"kubernetes.io/projected/6930710a-98d4-4622-b0f1-a724f5996bc0-kube-api-access-hl4th\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 17:58:49.676391 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.676356    2570 generic.go:356] "Generic (PLEG): container finished" podID="191b19b3-58ad-48c5-97de-08d3a61feb5b" containerID="4b63e08b9a72d64ceacb71c07481caf09b8d668e8deca10121c830560bed0e44" exitCode=0
Jul 26 17:58:49.676824 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.676400    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-pt4bf"
Jul 26 17:58:49.676824 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.676440    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-pt4bf" event={"ID":"191b19b3-58ad-48c5-97de-08d3a61feb5b","Type":"ContainerDied","Data":"4b63e08b9a72d64ceacb71c07481caf09b8d668e8deca10121c830560bed0e44"}
Jul 26 17:58:49.676824 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.676484    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-pt4bf" event={"ID":"191b19b3-58ad-48c5-97de-08d3a61feb5b","Type":"ContainerDied","Data":"017464c5b9b1d17c6471c612e6edba38ec167fe07d3b0861e932c4851826ba9c"}
Jul 26 17:58:49.676824 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.676504    2570 scope.go:117] "RemoveContainer" containerID="4b63e08b9a72d64ceacb71c07481caf09b8d668e8deca10121c830560bed0e44"
Jul 26 17:58:49.677767 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.677748    2570 generic.go:356] "Generic (PLEG): container finished" podID="6930710a-98d4-4622-b0f1-a724f5996bc0" containerID="93066599034f4b087b957e61bbf7e81fa6522eeeb64a30b0124394b0ac8f4a99" exitCode=0
Jul 26 17:58:49.677828 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.677792    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4cjbv"
Jul 26 17:58:49.677882 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.677829    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4cjbv" event={"ID":"6930710a-98d4-4622-b0f1-a724f5996bc0","Type":"ContainerDied","Data":"93066599034f4b087b957e61bbf7e81fa6522eeeb64a30b0124394b0ac8f4a99"}
Jul 26 17:58:49.677882 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.677873    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4cjbv" event={"ID":"6930710a-98d4-4622-b0f1-a724f5996bc0","Type":"ContainerDied","Data":"c20d088ffcaf73230756eda5b3a2e44ae3850c1282a938336c4a1f2e0426a623"}
Jul 26 17:58:49.685252 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.685235    2570 scope.go:117] "RemoveContainer" containerID="4b63e08b9a72d64ceacb71c07481caf09b8d668e8deca10121c830560bed0e44"
Jul 26 17:58:49.685514 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:58:49.685495    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b63e08b9a72d64ceacb71c07481caf09b8d668e8deca10121c830560bed0e44\": container with ID starting with 4b63e08b9a72d64ceacb71c07481caf09b8d668e8deca10121c830560bed0e44 not found: ID does not exist" containerID="4b63e08b9a72d64ceacb71c07481caf09b8d668e8deca10121c830560bed0e44"
Jul 26 17:58:49.685575 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.685522    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b63e08b9a72d64ceacb71c07481caf09b8d668e8deca10121c830560bed0e44"} err="failed to get container status \"4b63e08b9a72d64ceacb71c07481caf09b8d668e8deca10121c830560bed0e44\": rpc error: code = NotFound desc = could not find container \"4b63e08b9a72d64ceacb71c07481caf09b8d668e8deca10121c830560bed0e44\": container with ID starting with 4b63e08b9a72d64ceacb71c07481caf09b8d668e8deca10121c830560bed0e44 not found: ID does not exist"
Jul 26 17:58:49.685575 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.685538    2570 scope.go:117] "RemoveContainer" containerID="93066599034f4b087b957e61bbf7e81fa6522eeeb64a30b0124394b0ac8f4a99"
Jul 26 17:58:49.693388 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.693374    2570 scope.go:117] "RemoveContainer" containerID="93066599034f4b087b957e61bbf7e81fa6522eeeb64a30b0124394b0ac8f4a99"
Jul 26 17:58:49.693617 ip-10-0-128-45 kubenswrapper[2570]: E0726 17:58:49.693603    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93066599034f4b087b957e61bbf7e81fa6522eeeb64a30b0124394b0ac8f4a99\": container with ID starting with 93066599034f4b087b957e61bbf7e81fa6522eeeb64a30b0124394b0ac8f4a99 not found: ID does not exist" containerID="93066599034f4b087b957e61bbf7e81fa6522eeeb64a30b0124394b0ac8f4a99"
Jul 26 17:58:49.693660 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.693622    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93066599034f4b087b957e61bbf7e81fa6522eeeb64a30b0124394b0ac8f4a99"} err="failed to get container status \"93066599034f4b087b957e61bbf7e81fa6522eeeb64a30b0124394b0ac8f4a99\": rpc error: code = NotFound desc = could not find container \"93066599034f4b087b957e61bbf7e81fa6522eeeb64a30b0124394b0ac8f4a99\": container with ID starting with 93066599034f4b087b957e61bbf7e81fa6522eeeb64a30b0124394b0ac8f4a99 not found: ID does not exist"
Jul 26 17:58:49.700835 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.700816    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pt4bf"]
Jul 26 17:58:49.705050 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.705028    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pt4bf"]
Jul 26 17:58:49.714373 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.714344    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4cjbv"]
Jul 26 17:58:49.718134 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.718100    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4cjbv"]
Jul 26 17:58:49.838166 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.838141    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="191b19b3-58ad-48c5-97de-08d3a61feb5b" path="/var/lib/kubelet/pods/191b19b3-58ad-48c5-97de-08d3a61feb5b/volumes"
Jul 26 17:58:49.838448 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.838436    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6930710a-98d4-4622-b0f1-a724f5996bc0" path="/var/lib/kubelet/pods/6930710a-98d4-4622-b0f1-a724f5996bc0/volumes"
Jul 26 17:58:49.838709 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:58:49.838698    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e4d3bfd-0159-40c6-89ce-16d411a1574c" path="/var/lib/kubelet/pods/6e4d3bfd-0159-40c6-89ce-16d411a1574c/volumes"
Jul 26 17:59:03.801666 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:03.801638    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 17:59:03.808960 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:03.808928    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 17:59:30.924578 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:30.924497    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-664bf9fcf6-np9qj"]
Jul 26 17:59:30.927158 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:30.924818    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6930710a-98d4-4622-b0f1-a724f5996bc0" containerName="authorino"
Jul 26 17:59:30.927158 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:30.924829    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6930710a-98d4-4622-b0f1-a724f5996bc0" containerName="authorino"
Jul 26 17:59:30.927158 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:30.924836    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="191b19b3-58ad-48c5-97de-08d3a61feb5b" containerName="authorino"
Jul 26 17:59:30.927158 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:30.924843    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="191b19b3-58ad-48c5-97de-08d3a61feb5b" containerName="authorino"
Jul 26 17:59:30.927158 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:30.924896    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="6930710a-98d4-4622-b0f1-a724f5996bc0" containerName="authorino"
Jul 26 17:59:30.927158 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:30.924907    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="191b19b3-58ad-48c5-97de-08d3a61feb5b" containerName="authorino"
Jul 26 17:59:30.927966 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:30.927950    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-664bf9fcf6-np9qj"
Jul 26 17:59:30.930151 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:30.930108    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 26 17:59:30.931007 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:30.930981    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 26 17:59:30.931007 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:30.931000    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-qb2nc\""
Jul 26 17:59:30.931182 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:30.931005    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 17:59:30.938481 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:30.938461    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-664bf9fcf6-np9qj"]
Jul 26 17:59:30.956588 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:30.956552    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae329aad-5450-45d6-9c7f-82f6ff9e32d6-openshift-service-ca\") pod \"maas-api-664bf9fcf6-np9qj\" (UID: \"ae329aad-5450-45d6-9c7f-82f6ff9e32d6\") " pod="odh-ai-gateway-infra/maas-api-664bf9fcf6-np9qj"
Jul 26 17:59:30.956710 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:30.956594    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxs74\" (UniqueName: \"kubernetes.io/projected/ae329aad-5450-45d6-9c7f-82f6ff9e32d6-kube-api-access-hxs74\") pod \"maas-api-664bf9fcf6-np9qj\" (UID: \"ae329aad-5450-45d6-9c7f-82f6ff9e32d6\") " pod="odh-ai-gateway-infra/maas-api-664bf9fcf6-np9qj"
Jul 26 17:59:30.956710 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:30.956673    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae329aad-5450-45d6-9c7f-82f6ff9e32d6-maas-api-tls\") pod \"maas-api-664bf9fcf6-np9qj\" (UID: \"ae329aad-5450-45d6-9c7f-82f6ff9e32d6\") " pod="odh-ai-gateway-infra/maas-api-664bf9fcf6-np9qj"
Jul 26 17:59:31.057974 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.057940    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae329aad-5450-45d6-9c7f-82f6ff9e32d6-maas-api-tls\") pod \"maas-api-664bf9fcf6-np9qj\" (UID: \"ae329aad-5450-45d6-9c7f-82f6ff9e32d6\") " pod="odh-ai-gateway-infra/maas-api-664bf9fcf6-np9qj"
Jul 26 17:59:31.058177 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.057999    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae329aad-5450-45d6-9c7f-82f6ff9e32d6-openshift-service-ca\") pod \"maas-api-664bf9fcf6-np9qj\" (UID: \"ae329aad-5450-45d6-9c7f-82f6ff9e32d6\") " pod="odh-ai-gateway-infra/maas-api-664bf9fcf6-np9qj"
Jul 26 17:59:31.058177 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.058028    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxs74\" (UniqueName: \"kubernetes.io/projected/ae329aad-5450-45d6-9c7f-82f6ff9e32d6-kube-api-access-hxs74\") pod \"maas-api-664bf9fcf6-np9qj\" (UID: \"ae329aad-5450-45d6-9c7f-82f6ff9e32d6\") " pod="odh-ai-gateway-infra/maas-api-664bf9fcf6-np9qj"
Jul 26 17:59:31.058683 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.058657    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae329aad-5450-45d6-9c7f-82f6ff9e32d6-openshift-service-ca\") pod \"maas-api-664bf9fcf6-np9qj\" (UID: \"ae329aad-5450-45d6-9c7f-82f6ff9e32d6\") " pod="odh-ai-gateway-infra/maas-api-664bf9fcf6-np9qj"
Jul 26 17:59:31.060505 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.060481    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae329aad-5450-45d6-9c7f-82f6ff9e32d6-maas-api-tls\") pod \"maas-api-664bf9fcf6-np9qj\" (UID: \"ae329aad-5450-45d6-9c7f-82f6ff9e32d6\") " pod="odh-ai-gateway-infra/maas-api-664bf9fcf6-np9qj"
Jul 26 17:59:31.068032 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.068008    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxs74\" (UniqueName: \"kubernetes.io/projected/ae329aad-5450-45d6-9c7f-82f6ff9e32d6-kube-api-access-hxs74\") pod \"maas-api-664bf9fcf6-np9qj\" (UID: \"ae329aad-5450-45d6-9c7f-82f6ff9e32d6\") " pod="odh-ai-gateway-infra/maas-api-664bf9fcf6-np9qj"
Jul 26 17:59:31.239679 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.239588    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-664bf9fcf6-np9qj"
Jul 26 17:59:31.376405 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.376377    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-664bf9fcf6-np9qj"]
Jul 26 17:59:31.378050 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:59:31.378020    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae329aad_5450_45d6_9c7f_82f6ff9e32d6.slice/crio-ec834146bba0f7599eb95d214c1fef63d6a5c6fb3d22fae262f22bd9540cb4b8 WatchSource:0}: Error finding container ec834146bba0f7599eb95d214c1fef63d6a5c6fb3d22fae262f22bd9540cb4b8: Status 404 returned error can't find the container with id ec834146bba0f7599eb95d214c1fef63d6a5c6fb3d22fae262f22bd9540cb4b8
Jul 26 17:59:31.379438 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.379421    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 17:59:31.560771 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.560740    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-njrlb"]
Jul 26 17:59:31.565337 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.565306    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-njrlb"
Jul 26 17:59:31.568609 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.568591    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 26 17:59:31.568609 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.568601    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-kzzq7\""
Jul 26 17:59:31.573367 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.573346    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-njrlb"]
Jul 26 17:59:31.601805 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.601772    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-fw8n6"]
Jul 26 17:59:31.605336 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.605313    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-fw8n6"
Jul 26 17:59:31.617070 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.617046    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-fw8n6"]
Jul 26 17:59:31.663290 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.663260    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/608f4b82-aaca-455f-9977-c820a04aea61-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-njrlb\" (UID: \"608f4b82-aaca-455f-9977-c820a04aea61\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-njrlb"
Jul 26 17:59:31.663461 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.663326    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22c0748d-5cd0-4d65-addb-2e9f3837a71d-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-fw8n6\" (UID: \"22c0748d-5cd0-4d65-addb-2e9f3837a71d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-fw8n6"
Jul 26 17:59:31.663461 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.663382    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrzl6\" (UniqueName: \"kubernetes.io/projected/22c0748d-5cd0-4d65-addb-2e9f3837a71d-kube-api-access-hrzl6\") pod \"payload-processing-5cc6cd444b-fw8n6\" (UID: \"22c0748d-5cd0-4d65-addb-2e9f3837a71d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-fw8n6"
Jul 26 17:59:31.663461 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.663405    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjwkh\" (UniqueName: \"kubernetes.io/projected/608f4b82-aaca-455f-9977-c820a04aea61-kube-api-access-sjwkh\") pod \"payload-pre-processing-687dbb6cf6-njrlb\" (UID: \"608f4b82-aaca-455f-9977-c820a04aea61\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-njrlb"
Jul 26 17:59:31.764357 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.764321    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22c0748d-5cd0-4d65-addb-2e9f3837a71d-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-fw8n6\" (UID: \"22c0748d-5cd0-4d65-addb-2e9f3837a71d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-fw8n6"
Jul 26 17:59:31.764565 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.764381    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrzl6\" (UniqueName: \"kubernetes.io/projected/22c0748d-5cd0-4d65-addb-2e9f3837a71d-kube-api-access-hrzl6\") pod \"payload-processing-5cc6cd444b-fw8n6\" (UID: \"22c0748d-5cd0-4d65-addb-2e9f3837a71d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-fw8n6"
Jul 26 17:59:31.764565 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.764402    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjwkh\" (UniqueName: \"kubernetes.io/projected/608f4b82-aaca-455f-9977-c820a04aea61-kube-api-access-sjwkh\") pod \"payload-pre-processing-687dbb6cf6-njrlb\" (UID: \"608f4b82-aaca-455f-9977-c820a04aea61\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-njrlb"
Jul 26 17:59:31.764565 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.764425    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/608f4b82-aaca-455f-9977-c820a04aea61-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-njrlb\" (UID: \"608f4b82-aaca-455f-9977-c820a04aea61\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-njrlb"
Jul 26 17:59:31.765187 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.765163    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/608f4b82-aaca-455f-9977-c820a04aea61-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-njrlb\" (UID: \"608f4b82-aaca-455f-9977-c820a04aea61\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-njrlb"
Jul 26 17:59:31.765285 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.765164    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22c0748d-5cd0-4d65-addb-2e9f3837a71d-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-fw8n6\" (UID: \"22c0748d-5cd0-4d65-addb-2e9f3837a71d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-fw8n6"
Jul 26 17:59:31.773065 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.773041    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjwkh\" (UniqueName: \"kubernetes.io/projected/608f4b82-aaca-455f-9977-c820a04aea61-kube-api-access-sjwkh\") pod \"payload-pre-processing-687dbb6cf6-njrlb\" (UID: \"608f4b82-aaca-455f-9977-c820a04aea61\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-njrlb"
Jul 26 17:59:31.773175 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.773076    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrzl6\" (UniqueName: \"kubernetes.io/projected/22c0748d-5cd0-4d65-addb-2e9f3837a71d-kube-api-access-hrzl6\") pod \"payload-processing-5cc6cd444b-fw8n6\" (UID: \"22c0748d-5cd0-4d65-addb-2e9f3837a71d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-fw8n6"
Jul 26 17:59:31.877140 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.877037    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-njrlb"
Jul 26 17:59:31.892034 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.892002    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-664bf9fcf6-np9qj" event={"ID":"ae329aad-5450-45d6-9c7f-82f6ff9e32d6","Type":"ContainerStarted","Data":"ec834146bba0f7599eb95d214c1fef63d6a5c6fb3d22fae262f22bd9540cb4b8"}
Jul 26 17:59:31.915182 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:31.915148    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-fw8n6"
Jul 26 17:59:32.034774 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:32.034742    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-njrlb"]
Jul 26 17:59:32.037598 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:59:32.037568    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod608f4b82_aaca_455f_9977_c820a04aea61.slice/crio-03b3b5ceec5e3becf4565bc80bd1ed4db193fbe1da7c09c1b2b004952390be9d WatchSource:0}: Error finding container 03b3b5ceec5e3becf4565bc80bd1ed4db193fbe1da7c09c1b2b004952390be9d: Status 404 returned error can't find the container with id 03b3b5ceec5e3becf4565bc80bd1ed4db193fbe1da7c09c1b2b004952390be9d
Jul 26 17:59:32.093163 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:32.093134    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-fw8n6"]
Jul 26 17:59:32.096522 ip-10-0-128-45 kubenswrapper[2570]: W0726 17:59:32.096459    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22c0748d_5cd0_4d65_addb_2e9f3837a71d.slice/crio-0ebb22d36b04e07051e3ebba177b9cf8ae16084240963529d87387482f4788de WatchSource:0}: Error finding container 0ebb22d36b04e07051e3ebba177b9cf8ae16084240963529d87387482f4788de: Status 404 returned error can't find the container with id 0ebb22d36b04e07051e3ebba177b9cf8ae16084240963529d87387482f4788de
Jul 26 17:59:32.901492 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:32.901363    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-fw8n6" event={"ID":"22c0748d-5cd0-4d65-addb-2e9f3837a71d","Type":"ContainerStarted","Data":"0ebb22d36b04e07051e3ebba177b9cf8ae16084240963529d87387482f4788de"}
Jul 26 17:59:32.903195 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:32.903123    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-njrlb" event={"ID":"608f4b82-aaca-455f-9977-c820a04aea61","Type":"ContainerStarted","Data":"03b3b5ceec5e3becf4565bc80bd1ed4db193fbe1da7c09c1b2b004952390be9d"}
Jul 26 17:59:34.916804 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:34.916771    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-fw8n6" event={"ID":"22c0748d-5cd0-4d65-addb-2e9f3837a71d","Type":"ContainerStarted","Data":"5346bc39e75a7cb6fdeb5bb6a244816e098d9008abe70ae1f47141af2f69454e"}
Jul 26 17:59:34.917282 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:34.916830    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-fw8n6"
Jul 26 17:59:34.918345 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:34.918311    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-664bf9fcf6-np9qj" event={"ID":"ae329aad-5450-45d6-9c7f-82f6ff9e32d6","Type":"ContainerStarted","Data":"fa3ba30a0dc3e5c3b72a5f250148a3b9bf1a6ac2b32ef15ae5ca0d073a632282"}
Jul 26 17:59:34.918458 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:34.918363    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-664bf9fcf6-np9qj"
Jul 26 17:59:34.919800 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:34.919780    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-njrlb" event={"ID":"608f4b82-aaca-455f-9977-c820a04aea61","Type":"ContainerStarted","Data":"563a090e09cbb24a98b339c6122097b2d3b64e48cd8b0847c917ccc72032b6f4"}
Jul 26 17:59:34.919909 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:34.919894    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-njrlb"
Jul 26 17:59:34.934418 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:34.934370    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-fw8n6" podStartSLOduration=1.662233758 podStartE2EDuration="3.934351785s" totalImagesPullingTime="2.272118027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:59:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:59:34.931888946 +0000 UTC m=+631.584079820" watchObservedRunningTime="2026-07-26 17:59:34.934351785 +0000 UTC m=+631.586542644"
Jul 26 17:59:34.947491 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:34.947451    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-664bf9fcf6-np9qj" podStartSLOduration=1.962682592 podStartE2EDuration="4.94744099s" totalImagesPullingTime="2.984758398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:59:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:59:34.946917723 +0000 UTC m=+631.599108583" watchObservedRunningTime="2026-07-26 17:59:34.94744099 +0000 UTC m=+631.599631847"
Jul 26 17:59:34.963067 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:34.963026    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-njrlb" podStartSLOduration=1.638542267 podStartE2EDuration="3.963013809s" totalImagesPullingTime="2.324471542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:59:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:59:34.960666746 +0000 UTC m=+631.612857606" watchObservedRunningTime="2026-07-26 17:59:34.963013809 +0000 UTC m=+631.615204667"
Jul 26 17:59:40.931235 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:40.931207    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-664bf9fcf6-np9qj"
Jul 26 17:59:45.927636 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:45.927605    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-fw8n6"
Jul 26 17:59:45.928027 ip-10-0-128-45 kubenswrapper[2570]: I0726 17:59:45.927683    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-njrlb"
Jul 26 18:00:00.133814 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:00.133781    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751480-gzjtm"]
Jul 26 18:00:00.137028 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:00.137011    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751480-gzjtm"
Jul 26 18:00:00.139446 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:00.139427    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-swzk5\""
Jul 26 18:00:00.145062 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:00.145038    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751480-gzjtm"]
Jul 26 18:00:00.149100 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:00.149076    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751480-gzjtm"
Jul 26 18:00:00.293272 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:00.293248    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751480-gzjtm"]
Jul 26 18:00:00.295769 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:00:00.295738    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86890ed7_3503_454a_b6e9_8805d925ff27.slice/crio-7fb8b6fc9fc441599240ace9fc3806c65a18c6d0eadd3a149e125bfedbd4256d WatchSource:0}: Error finding container 7fb8b6fc9fc441599240ace9fc3806c65a18c6d0eadd3a149e125bfedbd4256d: Status 404 returned error can't find the container with id 7fb8b6fc9fc441599240ace9fc3806c65a18c6d0eadd3a149e125bfedbd4256d
Jul 26 18:00:01.052718 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:01.052683    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751480-gzjtm" event={"ID":"86890ed7-3503-454a-b6e9-8805d925ff27","Type":"ContainerStarted","Data":"7fb8b6fc9fc441599240ace9fc3806c65a18c6d0eadd3a149e125bfedbd4256d"}
Jul 26 18:00:04.071701 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:04.071667    2570 generic.go:356] "Generic (PLEG): container finished" podID="86890ed7-3503-454a-b6e9-8805d925ff27" containerID="171ae656604e663cb03dc2d17706bb67290ace0269b1814e8279957eb4f75337" exitCode=0
Jul 26 18:00:04.072072 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:04.071746    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751480-gzjtm" event={"ID":"86890ed7-3503-454a-b6e9-8805d925ff27","Type":"ContainerDied","Data":"171ae656604e663cb03dc2d17706bb67290ace0269b1814e8279957eb4f75337"}
Jul 26 18:00:05.208252 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:05.208230    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751480-gzjtm"
Jul 26 18:00:06.082966 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:06.082932    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751480-gzjtm"
Jul 26 18:00:06.083206 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:06.082966    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751480-gzjtm" event={"ID":"86890ed7-3503-454a-b6e9-8805d925ff27","Type":"ContainerDied","Data":"7fb8b6fc9fc441599240ace9fc3806c65a18c6d0eadd3a149e125bfedbd4256d"}
Jul 26 18:00:06.083206 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:06.083010    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7fb8b6fc9fc441599240ace9fc3806c65a18c6d0eadd3a149e125bfedbd4256d"
Jul 26 18:00:28.594410 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.594375    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"]
Jul 26 18:00:28.594797 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.594730    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86890ed7-3503-454a-b6e9-8805d925ff27" containerName="cleanup"
Jul 26 18:00:28.594797 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.594740    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="86890ed7-3503-454a-b6e9-8805d925ff27" containerName="cleanup"
Jul 26 18:00:28.594872 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.594798    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="86890ed7-3503-454a-b6e9-8805d925ff27" containerName="cleanup"
Jul 26 18:00:28.597796 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.597780    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"
Jul 26 18:00:28.599757 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.599738    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 26 18:00:28.600490 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.600468    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 26 18:00:28.600622 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.600602    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-8klrd\""
Jul 26 18:00:28.600691 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.600665    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 26 18:00:28.606982 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.606958    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"]
Jul 26 18:00:28.647887 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.647850    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4efb9111-d468-40bb-a3c8-8f3504934007-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mfl8j\" (UID: \"4efb9111-d468-40bb-a3c8-8f3504934007\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"
Jul 26 18:00:28.647887 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.647893    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4efb9111-d468-40bb-a3c8-8f3504934007-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mfl8j\" (UID: \"4efb9111-d468-40bb-a3c8-8f3504934007\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"
Jul 26 18:00:28.648097 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.647915    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4efb9111-d468-40bb-a3c8-8f3504934007-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mfl8j\" (UID: \"4efb9111-d468-40bb-a3c8-8f3504934007\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"
Jul 26 18:00:28.648097 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.648018    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4efb9111-d468-40bb-a3c8-8f3504934007-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mfl8j\" (UID: \"4efb9111-d468-40bb-a3c8-8f3504934007\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"
Jul 26 18:00:28.648097 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.648070    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4efb9111-d468-40bb-a3c8-8f3504934007-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mfl8j\" (UID: \"4efb9111-d468-40bb-a3c8-8f3504934007\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"
Jul 26 18:00:28.648245 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.648105    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v4df\" (UniqueName: \"kubernetes.io/projected/4efb9111-d468-40bb-a3c8-8f3504934007-kube-api-access-7v4df\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mfl8j\" (UID: \"4efb9111-d468-40bb-a3c8-8f3504934007\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"
Jul 26 18:00:28.748968 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.748932    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4efb9111-d468-40bb-a3c8-8f3504934007-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mfl8j\" (UID: \"4efb9111-d468-40bb-a3c8-8f3504934007\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"
Jul 26 18:00:28.749198 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.748981    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4efb9111-d468-40bb-a3c8-8f3504934007-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mfl8j\" (UID: \"4efb9111-d468-40bb-a3c8-8f3504934007\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"
Jul 26 18:00:28.749198 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.749008    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4efb9111-d468-40bb-a3c8-8f3504934007-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mfl8j\" (UID: \"4efb9111-d468-40bb-a3c8-8f3504934007\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"
Jul 26 18:00:28.749198 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.749033    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4efb9111-d468-40bb-a3c8-8f3504934007-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mfl8j\" (UID: \"4efb9111-d468-40bb-a3c8-8f3504934007\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"
Jul 26 18:00:28.749198 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.749060    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4efb9111-d468-40bb-a3c8-8f3504934007-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mfl8j\" (UID: \"4efb9111-d468-40bb-a3c8-8f3504934007\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"
Jul 26 18:00:28.749198 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.749098    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v4df\" (UniqueName: \"kubernetes.io/projected/4efb9111-d468-40bb-a3c8-8f3504934007-kube-api-access-7v4df\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mfl8j\" (UID: \"4efb9111-d468-40bb-a3c8-8f3504934007\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"
Jul 26 18:00:28.749497 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.749445    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4efb9111-d468-40bb-a3c8-8f3504934007-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mfl8j\" (UID: \"4efb9111-d468-40bb-a3c8-8f3504934007\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"
Jul 26 18:00:28.749497 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.749454    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4efb9111-d468-40bb-a3c8-8f3504934007-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mfl8j\" (UID: \"4efb9111-d468-40bb-a3c8-8f3504934007\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"
Jul 26 18:00:28.749602 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.749514    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4efb9111-d468-40bb-a3c8-8f3504934007-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mfl8j\" (UID: \"4efb9111-d468-40bb-a3c8-8f3504934007\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"
Jul 26 18:00:28.751507 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.751486    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4efb9111-d468-40bb-a3c8-8f3504934007-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mfl8j\" (UID: \"4efb9111-d468-40bb-a3c8-8f3504934007\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"
Jul 26 18:00:28.751694 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.751674    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4efb9111-d468-40bb-a3c8-8f3504934007-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mfl8j\" (UID: \"4efb9111-d468-40bb-a3c8-8f3504934007\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"
Jul 26 18:00:28.757250 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.757231    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v4df\" (UniqueName: \"kubernetes.io/projected/4efb9111-d468-40bb-a3c8-8f3504934007-kube-api-access-7v4df\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mfl8j\" (UID: \"4efb9111-d468-40bb-a3c8-8f3504934007\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"
Jul 26 18:00:28.908940 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:28.908848    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"
Jul 26 18:00:29.046344 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:29.046317    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"]
Jul 26 18:00:29.048214 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:00:29.048187    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4efb9111_d468_40bb_a3c8_8f3504934007.slice/crio-1e13d90667fbe706433687a1e9ff9ce1a76080ed3c0dfc3af78a65275303fb67 WatchSource:0}: Error finding container 1e13d90667fbe706433687a1e9ff9ce1a76080ed3c0dfc3af78a65275303fb67: Status 404 returned error can't find the container with id 1e13d90667fbe706433687a1e9ff9ce1a76080ed3c0dfc3af78a65275303fb67
Jul 26 18:00:29.201987 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:29.201895    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j" event={"ID":"4efb9111-d468-40bb-a3c8-8f3504934007","Type":"ContainerStarted","Data":"1e13d90667fbe706433687a1e9ff9ce1a76080ed3c0dfc3af78a65275303fb67"}
Jul 26 18:00:33.226059 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:33.226012    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j" event={"ID":"4efb9111-d468-40bb-a3c8-8f3504934007","Type":"ContainerStarted","Data":"0b2c6aa158daa3d810f9c203444fa43fbcacf408e01e7145fb50cf56b90e6c62"}
Jul 26 18:00:33.244758 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:33.244697    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j" podStartSLOduration=1.678236773 podStartE2EDuration="5.244678824s" totalImagesPullingTime="3.566442051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:00:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:00:33.243296223 +0000 UTC m=+689.895487081" watchObservedRunningTime="2026-07-26 18:00:33.244678824 +0000 UTC m=+689.896869682"
Jul 26 18:00:38.909513 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:38.909469    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"
Jul 26 18:00:38.909999 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:38.909534    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"
Jul 26 18:00:38.922238 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:38.922214    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"
Jul 26 18:00:39.270366 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:00:39.270337    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mfl8j"
Jul 26 18:02:13.674010 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:13.673980    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76ff7f7cbc-zzzx2"]
Jul 26 18:02:13.677263 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:13.677246    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76ff7f7cbc-zzzx2"
Jul 26 18:02:13.686239 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:13.686212    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76ff7f7cbc-zzzx2"]
Jul 26 18:02:13.758335 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:13.758300    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1e33d04d-5a1e-48b3-9fd1-c1c61cb2fdaf-tls-cert\") pod \"authorino-76ff7f7cbc-zzzx2\" (UID: \"1e33d04d-5a1e-48b3-9fd1-c1c61cb2fdaf\") " pod="kuadrant-system/authorino-76ff7f7cbc-zzzx2"
Jul 26 18:02:13.758508 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:13.758351    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5jtw\" (UniqueName: \"kubernetes.io/projected/1e33d04d-5a1e-48b3-9fd1-c1c61cb2fdaf-kube-api-access-r5jtw\") pod \"authorino-76ff7f7cbc-zzzx2\" (UID: \"1e33d04d-5a1e-48b3-9fd1-c1c61cb2fdaf\") " pod="kuadrant-system/authorino-76ff7f7cbc-zzzx2"
Jul 26 18:02:13.858891 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:13.858859    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5jtw\" (UniqueName: \"kubernetes.io/projected/1e33d04d-5a1e-48b3-9fd1-c1c61cb2fdaf-kube-api-access-r5jtw\") pod \"authorino-76ff7f7cbc-zzzx2\" (UID: \"1e33d04d-5a1e-48b3-9fd1-c1c61cb2fdaf\") " pod="kuadrant-system/authorino-76ff7f7cbc-zzzx2"
Jul 26 18:02:13.859058 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:13.858932    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1e33d04d-5a1e-48b3-9fd1-c1c61cb2fdaf-tls-cert\") pod \"authorino-76ff7f7cbc-zzzx2\" (UID: \"1e33d04d-5a1e-48b3-9fd1-c1c61cb2fdaf\") " pod="kuadrant-system/authorino-76ff7f7cbc-zzzx2"
Jul 26 18:02:13.861353 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:13.861336    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1e33d04d-5a1e-48b3-9fd1-c1c61cb2fdaf-tls-cert\") pod \"authorino-76ff7f7cbc-zzzx2\" (UID: \"1e33d04d-5a1e-48b3-9fd1-c1c61cb2fdaf\") " pod="kuadrant-system/authorino-76ff7f7cbc-zzzx2"
Jul 26 18:02:13.866704 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:13.866682    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5jtw\" (UniqueName: \"kubernetes.io/projected/1e33d04d-5a1e-48b3-9fd1-c1c61cb2fdaf-kube-api-access-r5jtw\") pod \"authorino-76ff7f7cbc-zzzx2\" (UID: \"1e33d04d-5a1e-48b3-9fd1-c1c61cb2fdaf\") " pod="kuadrant-system/authorino-76ff7f7cbc-zzzx2"
Jul 26 18:02:13.986578 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:13.986496    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76ff7f7cbc-zzzx2"
Jul 26 18:02:14.135217 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:14.135178    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76ff7f7cbc-zzzx2"]
Jul 26 18:02:14.139991 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:02:14.139962    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e33d04d_5a1e_48b3_9fd1_c1c61cb2fdaf.slice/crio-5fcfec3ebf9f4bf2f1c9b8e563f562b50b259828c73eef79670b02537135ed3b WatchSource:0}: Error finding container 5fcfec3ebf9f4bf2f1c9b8e563f562b50b259828c73eef79670b02537135ed3b: Status 404 returned error can't find the container with id 5fcfec3ebf9f4bf2f1c9b8e563f562b50b259828c73eef79670b02537135ed3b
Jul 26 18:02:14.757696 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:14.757660    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76ff7f7cbc-zzzx2" event={"ID":"1e33d04d-5a1e-48b3-9fd1-c1c61cb2fdaf","Type":"ContainerStarted","Data":"7cbc811c97007823ebf9f466ee571a6cfb2ff9e7c561aca2088f525d7c557259"}
Jul 26 18:02:14.757696 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:14.757702    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76ff7f7cbc-zzzx2" event={"ID":"1e33d04d-5a1e-48b3-9fd1-c1c61cb2fdaf","Type":"ContainerStarted","Data":"5fcfec3ebf9f4bf2f1c9b8e563f562b50b259828c73eef79670b02537135ed3b"}
Jul 26 18:02:14.772853 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:14.772788    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-76ff7f7cbc-zzzx2" podStartSLOduration=1.356702212 podStartE2EDuration="1.772771483s" totalImagesPullingTime="416.069271ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:02:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:02:14.771240986 +0000 UTC m=+791.423431847" watchObservedRunningTime="2026-07-26 18:02:14.772771483 +0000 UTC m=+791.424962341"
Jul 26 18:02:14.804138 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:14.803684    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b758bfb7d-fdlfz"]
Jul 26 18:02:14.804138 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:14.803897    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7b758bfb7d-fdlfz" podUID="1f1c1680-70be-439c-a8b6-5d63fffd6704" containerName="authorino" containerID="cri-o://d6339c07341812cbef97ed92eebf5ecd6c02924dff8175d63939a196f1e252f2" gracePeriod=30
Jul 26 18:02:15.061282 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:15.061254    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b758bfb7d-fdlfz"
Jul 26 18:02:15.067855 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:15.067834    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwqqm\" (UniqueName: \"kubernetes.io/projected/1f1c1680-70be-439c-a8b6-5d63fffd6704-kube-api-access-kwqqm\") pod \"1f1c1680-70be-439c-a8b6-5d63fffd6704\" (UID: \"1f1c1680-70be-439c-a8b6-5d63fffd6704\") "
Jul 26 18:02:15.067933 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:15.067886    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1f1c1680-70be-439c-a8b6-5d63fffd6704-tls-cert\") pod \"1f1c1680-70be-439c-a8b6-5d63fffd6704\" (UID: \"1f1c1680-70be-439c-a8b6-5d63fffd6704\") "
Jul 26 18:02:15.069888 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:15.069860    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f1c1680-70be-439c-a8b6-5d63fffd6704-kube-api-access-kwqqm" (OuterVolumeSpecName: "kube-api-access-kwqqm") pod "1f1c1680-70be-439c-a8b6-5d63fffd6704" (UID: "1f1c1680-70be-439c-a8b6-5d63fffd6704"). InnerVolumeSpecName "kube-api-access-kwqqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:02:15.079818 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:15.079785    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f1c1680-70be-439c-a8b6-5d63fffd6704-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "1f1c1680-70be-439c-a8b6-5d63fffd6704" (UID: "1f1c1680-70be-439c-a8b6-5d63fffd6704"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:02:15.169239 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:15.169151    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwqqm\" (UniqueName: \"kubernetes.io/projected/1f1c1680-70be-439c-a8b6-5d63fffd6704-kube-api-access-kwqqm\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:02:15.169239 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:15.169186    2570 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1f1c1680-70be-439c-a8b6-5d63fffd6704-tls-cert\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:02:15.764435 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:15.764388    2570 generic.go:356] "Generic (PLEG): container finished" podID="1f1c1680-70be-439c-a8b6-5d63fffd6704" containerID="d6339c07341812cbef97ed92eebf5ecd6c02924dff8175d63939a196f1e252f2" exitCode=0
Jul 26 18:02:15.764435 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:15.764431    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b758bfb7d-fdlfz"
Jul 26 18:02:15.764983 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:15.764476    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b758bfb7d-fdlfz" event={"ID":"1f1c1680-70be-439c-a8b6-5d63fffd6704","Type":"ContainerDied","Data":"d6339c07341812cbef97ed92eebf5ecd6c02924dff8175d63939a196f1e252f2"}
Jul 26 18:02:15.764983 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:15.764513    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b758bfb7d-fdlfz" event={"ID":"1f1c1680-70be-439c-a8b6-5d63fffd6704","Type":"ContainerDied","Data":"13dc8c009af2d7f8ac58a76da301ab4ebdc23554b2aa43410c582db392562bf8"}
Jul 26 18:02:15.764983 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:15.764530    2570 scope.go:117] "RemoveContainer" containerID="d6339c07341812cbef97ed92eebf5ecd6c02924dff8175d63939a196f1e252f2"
Jul 26 18:02:15.775132 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:15.775092    2570 scope.go:117] "RemoveContainer" containerID="d6339c07341812cbef97ed92eebf5ecd6c02924dff8175d63939a196f1e252f2"
Jul 26 18:02:15.775451 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:02:15.775426    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6339c07341812cbef97ed92eebf5ecd6c02924dff8175d63939a196f1e252f2\": container with ID starting with d6339c07341812cbef97ed92eebf5ecd6c02924dff8175d63939a196f1e252f2 not found: ID does not exist" containerID="d6339c07341812cbef97ed92eebf5ecd6c02924dff8175d63939a196f1e252f2"
Jul 26 18:02:15.775550 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:15.775459    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6339c07341812cbef97ed92eebf5ecd6c02924dff8175d63939a196f1e252f2"} err="failed to get container status \"d6339c07341812cbef97ed92eebf5ecd6c02924dff8175d63939a196f1e252f2\": rpc error: code = NotFound desc = could not find container \"d6339c07341812cbef97ed92eebf5ecd6c02924dff8175d63939a196f1e252f2\": container with ID starting with d6339c07341812cbef97ed92eebf5ecd6c02924dff8175d63939a196f1e252f2 not found: ID does not exist"
Jul 26 18:02:15.800186 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:15.800148    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b758bfb7d-fdlfz"]
Jul 26 18:02:15.807362 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:15.807335    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7b758bfb7d-fdlfz"]
Jul 26 18:02:15.840465 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:02:15.840429    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f1c1680-70be-439c-a8b6-5d63fffd6704" path="/var/lib/kubelet/pods/1f1c1680-70be-439c-a8b6-5d63fffd6704/volumes"
Jul 26 18:04:03.841450 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:04:03.841376    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 18:04:03.843697 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:04:03.843673    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 18:09:03.878850 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:09:03.878816    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 18:09:03.879478 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:09:03.878921    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 18:14:03.911850 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:14:03.911822    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 18:14:03.913580 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:14:03.913557    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 18:15:00.151499 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:15:00.151455    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751495-8x728"]
Jul 26 18:15:00.152105 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:15:00.151973    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f1c1680-70be-439c-a8b6-5d63fffd6704" containerName="authorino"
Jul 26 18:15:00.152105 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:15:00.151993    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f1c1680-70be-439c-a8b6-5d63fffd6704" containerName="authorino"
Jul 26 18:15:00.152105 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:15:00.152102    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f1c1680-70be-439c-a8b6-5d63fffd6704" containerName="authorino"
Jul 26 18:15:00.155232 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:15:00.155212    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751495-8x728"
Jul 26 18:15:00.157882 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:15:00.157855    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-swzk5\""
Jul 26 18:15:00.163605 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:15:00.163580    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751495-8x728"]
Jul 26 18:15:00.166398 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:15:00.166379    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751495-8x728"
Jul 26 18:15:00.304479 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:15:00.304451    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751495-8x728"]
Jul 26 18:15:00.306018 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:15:00.305991    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23736da4_1b7c_4824_8a92_8a84a4db2a0a.slice/crio-c8bc00cad40913ebc748c4e986cca3efa12376579e6476bf24cfe3eb5cfd237f WatchSource:0}: Error finding container c8bc00cad40913ebc748c4e986cca3efa12376579e6476bf24cfe3eb5cfd237f: Status 404 returned error can't find the container with id c8bc00cad40913ebc748c4e986cca3efa12376579e6476bf24cfe3eb5cfd237f
Jul 26 18:15:00.789483 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:15:00.789442    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751495-8x728" event={"ID":"23736da4-1b7c-4824-8a92-8a84a4db2a0a","Type":"ContainerStarted","Data":"4301fb3faba1b348580cef1edafc78e3400bbb6a4c909fc5b45335352f1a6803"}
Jul 26 18:15:00.789681 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:15:00.789504    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751495-8x728" event={"ID":"23736da4-1b7c-4824-8a92-8a84a4db2a0a","Type":"ContainerStarted","Data":"c8bc00cad40913ebc748c4e986cca3efa12376579e6476bf24cfe3eb5cfd237f"}
Jul 26 18:15:00.807144 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:15:00.807029    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751495-8x728" podStartSLOduration=0.807013359 podStartE2EDuration="807.013359ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:15:00.806852594 +0000 UTC m=+1557.459043453" watchObservedRunningTime="2026-07-26 18:15:00.807013359 +0000 UTC m=+1557.459204217"
Jul 26 18:15:01.796695 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:15:01.796658    2570 generic.go:356] "Generic (PLEG): container finished" podID="23736da4-1b7c-4824-8a92-8a84a4db2a0a" containerID="4301fb3faba1b348580cef1edafc78e3400bbb6a4c909fc5b45335352f1a6803" exitCode=0
Jul 26 18:15:01.797080 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:15:01.796749    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751495-8x728" event={"ID":"23736da4-1b7c-4824-8a92-8a84a4db2a0a","Type":"ContainerDied","Data":"4301fb3faba1b348580cef1edafc78e3400bbb6a4c909fc5b45335352f1a6803"}
Jul 26 18:15:02.936178 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:15:02.936150    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751495-8x728"
Jul 26 18:15:03.809300 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:15:03.809261    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751495-8x728"
Jul 26 18:15:03.809300 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:15:03.809284    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751495-8x728" event={"ID":"23736da4-1b7c-4824-8a92-8a84a4db2a0a","Type":"ContainerDied","Data":"c8bc00cad40913ebc748c4e986cca3efa12376579e6476bf24cfe3eb5cfd237f"}
Jul 26 18:15:03.809519 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:15:03.809336    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8bc00cad40913ebc748c4e986cca3efa12376579e6476bf24cfe3eb5cfd237f"
Jul 26 18:17:25.078542 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.078503    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr"]
Jul 26 18:17:25.080806 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.079042    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23736da4-1b7c-4824-8a92-8a84a4db2a0a" containerName="cleanup"
Jul 26 18:17:25.080806 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.079058    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="23736da4-1b7c-4824-8a92-8a84a4db2a0a" containerName="cleanup"
Jul 26 18:17:25.080806 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.079160    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="23736da4-1b7c-4824-8a92-8a84a4db2a0a" containerName="cleanup"
Jul 26 18:17:25.082453 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.082434    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr"
Jul 26 18:17:25.084518 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.084494    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a0608e5c-serving-cert\""
Jul 26 18:17:25.094225 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.094201    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr"]
Jul 26 18:17:25.250001 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.249957    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ff7e4a9-d8f8-4778-b2c1-e318097c8e29-maas-api-tls\") pod \"maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr\" (UID: \"7ff7e4a9-d8f8-4778-b2c1-e318097c8e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr"
Jul 26 18:17:25.250245 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.250086    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m9c8\" (UniqueName: \"kubernetes.io/projected/7ff7e4a9-d8f8-4778-b2c1-e318097c8e29-kube-api-access-6m9c8\") pod \"maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr\" (UID: \"7ff7e4a9-d8f8-4778-b2c1-e318097c8e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr"
Jul 26 18:17:25.250245 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.250142    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff7e4a9-d8f8-4778-b2c1-e318097c8e29-openshift-service-ca\") pod \"maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr\" (UID: \"7ff7e4a9-d8f8-4778-b2c1-e318097c8e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr"
Jul 26 18:17:25.351302 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.351202    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ff7e4a9-d8f8-4778-b2c1-e318097c8e29-maas-api-tls\") pod \"maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr\" (UID: \"7ff7e4a9-d8f8-4778-b2c1-e318097c8e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr"
Jul 26 18:17:25.351302 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.351270    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m9c8\" (UniqueName: \"kubernetes.io/projected/7ff7e4a9-d8f8-4778-b2c1-e318097c8e29-kube-api-access-6m9c8\") pod \"maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr\" (UID: \"7ff7e4a9-d8f8-4778-b2c1-e318097c8e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr"
Jul 26 18:17:25.351302 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.351297    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff7e4a9-d8f8-4778-b2c1-e318097c8e29-openshift-service-ca\") pod \"maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr\" (UID: \"7ff7e4a9-d8f8-4778-b2c1-e318097c8e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr"
Jul 26 18:17:25.351900 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.351876    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff7e4a9-d8f8-4778-b2c1-e318097c8e29-openshift-service-ca\") pod \"maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr\" (UID: \"7ff7e4a9-d8f8-4778-b2c1-e318097c8e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr"
Jul 26 18:17:25.354061 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.354007    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ff7e4a9-d8f8-4778-b2c1-e318097c8e29-maas-api-tls\") pod \"maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr\" (UID: \"7ff7e4a9-d8f8-4778-b2c1-e318097c8e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr"
Jul 26 18:17:25.360434 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.360407    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m9c8\" (UniqueName: \"kubernetes.io/projected/7ff7e4a9-d8f8-4778-b2c1-e318097c8e29-kube-api-access-6m9c8\") pod \"maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr\" (UID: \"7ff7e4a9-d8f8-4778-b2c1-e318097c8e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr"
Jul 26 18:17:25.395195 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.395163    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr"
Jul 26 18:17:25.536570 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.536454    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr"]
Jul 26 18:17:25.539084 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:17:25.539055    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ff7e4a9_d8f8_4778_b2c1_e318097c8e29.slice/crio-b79a2627121dcfb840d952777d2e3667837024e0037dfecbaacbdbe07a8531d0 WatchSource:0}: Error finding container b79a2627121dcfb840d952777d2e3667837024e0037dfecbaacbdbe07a8531d0: Status 404 returned error can't find the container with id b79a2627121dcfb840d952777d2e3667837024e0037dfecbaacbdbe07a8531d0
Jul 26 18:17:25.540622 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.540599    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 18:17:25.555656 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.555622    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr" event={"ID":"7ff7e4a9-d8f8-4778-b2c1-e318097c8e29","Type":"ContainerStarted","Data":"b79a2627121dcfb840d952777d2e3667837024e0037dfecbaacbdbe07a8531d0"}
Jul 26 18:17:25.682388 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.682306    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m"]
Jul 26 18:17:25.687348 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.687323    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m"
Jul 26 18:17:25.689553 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.689521    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-a0608e5c\""
Jul 26 18:17:25.689698 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.689681    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-a0608e5c-dockercfg-xs6nf\""
Jul 26 18:17:25.698265 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.698220    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m"]
Jul 26 18:17:25.705204 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.705178    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6"]
Jul 26 18:17:25.708800 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.708784    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6"
Jul 26 18:17:25.719724 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.719700    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6"]
Jul 26 18:17:25.856204 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.856172    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dknhf\" (UniqueName: \"kubernetes.io/projected/1f1fdc55-bc8d-467c-b139-18392dda93cd-kube-api-access-dknhf\") pod \"payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6\" (UID: \"1f1fdc55-bc8d-467c-b139-18392dda93cd\") " pod="openshift-ingress/payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6"
Jul 26 18:17:25.856376 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.856217    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f1fdc55-bc8d-467c-b139-18392dda93cd-plugins-config-volume\") pod \"payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6\" (UID: \"1f1fdc55-bc8d-467c-b139-18392dda93cd\") " pod="openshift-ingress/payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6"
Jul 26 18:17:25.856376 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.856293    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lsll\" (UniqueName: \"kubernetes.io/projected/ecdb2082-167c-4571-a9a9-f2427c814a66-kube-api-access-5lsll\") pod \"payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m\" (UID: \"ecdb2082-167c-4571-a9a9-f2427c814a66\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m"
Jul 26 18:17:25.856376 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.856352    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecdb2082-167c-4571-a9a9-f2427c814a66-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m\" (UID: \"ecdb2082-167c-4571-a9a9-f2427c814a66\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m"
Jul 26 18:17:25.957573 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.957488    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lsll\" (UniqueName: \"kubernetes.io/projected/ecdb2082-167c-4571-a9a9-f2427c814a66-kube-api-access-5lsll\") pod \"payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m\" (UID: \"ecdb2082-167c-4571-a9a9-f2427c814a66\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m"
Jul 26 18:17:25.957573 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.957544    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecdb2082-167c-4571-a9a9-f2427c814a66-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m\" (UID: \"ecdb2082-167c-4571-a9a9-f2427c814a66\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m"
Jul 26 18:17:25.957813 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.957588    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dknhf\" (UniqueName: \"kubernetes.io/projected/1f1fdc55-bc8d-467c-b139-18392dda93cd-kube-api-access-dknhf\") pod \"payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6\" (UID: \"1f1fdc55-bc8d-467c-b139-18392dda93cd\") " pod="openshift-ingress/payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6"
Jul 26 18:17:25.957813 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.957620    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f1fdc55-bc8d-467c-b139-18392dda93cd-plugins-config-volume\") pod \"payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6\" (UID: \"1f1fdc55-bc8d-467c-b139-18392dda93cd\") " pod="openshift-ingress/payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6"
Jul 26 18:17:25.958417 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.958398    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f1fdc55-bc8d-467c-b139-18392dda93cd-plugins-config-volume\") pod \"payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6\" (UID: \"1f1fdc55-bc8d-467c-b139-18392dda93cd\") " pod="openshift-ingress/payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6"
Jul 26 18:17:25.958459 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.958408    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecdb2082-167c-4571-a9a9-f2427c814a66-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m\" (UID: \"ecdb2082-167c-4571-a9a9-f2427c814a66\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m"
Jul 26 18:17:25.965766 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.965737    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dknhf\" (UniqueName: \"kubernetes.io/projected/1f1fdc55-bc8d-467c-b139-18392dda93cd-kube-api-access-dknhf\") pod \"payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6\" (UID: \"1f1fdc55-bc8d-467c-b139-18392dda93cd\") " pod="openshift-ingress/payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6"
Jul 26 18:17:25.965766 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.965748    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lsll\" (UniqueName: \"kubernetes.io/projected/ecdb2082-167c-4571-a9a9-f2427c814a66-kube-api-access-5lsll\") pod \"payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m\" (UID: \"ecdb2082-167c-4571-a9a9-f2427c814a66\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m"
Jul 26 18:17:25.999604 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:25.999567    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m"
Jul 26 18:17:26.021459 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:26.021436    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6"
Jul 26 18:17:26.196206 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:26.196179    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6"]
Jul 26 18:17:26.197001 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:17:26.196975    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f1fdc55_bc8d_467c_b139_18392dda93cd.slice/crio-b7dae36034a5c8f2d7587d2c2832b1890091743cd69dc157974c7f5301e7209d WatchSource:0}: Error finding container b7dae36034a5c8f2d7587d2c2832b1890091743cd69dc157974c7f5301e7209d: Status 404 returned error can't find the container with id b7dae36034a5c8f2d7587d2c2832b1890091743cd69dc157974c7f5301e7209d
Jul 26 18:17:26.365125 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:26.365088    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m"]
Jul 26 18:17:26.366760 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:17:26.366717    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecdb2082_167c_4571_a9a9_f2427c814a66.slice/crio-ef91837a8bed05fdb7d3b396ac6bfbd95621c9f39e9ea883bd1a4c455da8905c WatchSource:0}: Error finding container ef91837a8bed05fdb7d3b396ac6bfbd95621c9f39e9ea883bd1a4c455da8905c: Status 404 returned error can't find the container with id ef91837a8bed05fdb7d3b396ac6bfbd95621c9f39e9ea883bd1a4c455da8905c
Jul 26 18:17:26.562650 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:26.562609    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m" event={"ID":"ecdb2082-167c-4571-a9a9-f2427c814a66","Type":"ContainerStarted","Data":"d463f4babd914b777872c58ca98abe915bd75730fec5c0e4cc23e5ee80b94636"}
Jul 26 18:17:26.562870 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:26.562676    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m" event={"ID":"ecdb2082-167c-4571-a9a9-f2427c814a66","Type":"ContainerStarted","Data":"ef91837a8bed05fdb7d3b396ac6bfbd95621c9f39e9ea883bd1a4c455da8905c"}
Jul 26 18:17:26.562870 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:26.562769    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m"
Jul 26 18:17:26.564311 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:26.564283    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr" event={"ID":"7ff7e4a9-d8f8-4778-b2c1-e318097c8e29","Type":"ContainerStarted","Data":"65c70b29b487473e718e3ddf08e0fa5db7fe8f1c534a029b04b7865bed619d99"}
Jul 26 18:17:26.564439 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:26.564385    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr"
Jul 26 18:17:26.565916 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:26.565894    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6" event={"ID":"1f1fdc55-bc8d-467c-b139-18392dda93cd","Type":"ContainerStarted","Data":"762c63168215e29307d396d8cfc472053ba88a4158d1d3fdcd3d55aced807b55"}
Jul 26 18:17:26.565997 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:26.565935    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6" event={"ID":"1f1fdc55-bc8d-467c-b139-18392dda93cd","Type":"ContainerStarted","Data":"b7dae36034a5c8f2d7587d2c2832b1890091743cd69dc157974c7f5301e7209d"}
Jul 26 18:17:26.566045 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:26.565996    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6"
Jul 26 18:17:26.578519 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:26.578460    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m" podStartSLOduration=1.5784423250000001 podStartE2EDuration="1.578442325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:17:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:17:26.576634898 +0000 UTC m=+1703.228825768" watchObservedRunningTime="2026-07-26 18:17:26.578442325 +0000 UTC m=+1703.230633185"
Jul 26 18:17:26.593650 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:26.593605    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr" podStartSLOduration=1.115969013 podStartE2EDuration="1.593589982s" totalImagesPullingTime="477.620969ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:17:26.592809468 +0000 UTC m=+1703.245000327" watchObservedRunningTime="2026-07-26 18:17:26.593589982 +0000 UTC m=+1703.245780840"
Jul 26 18:17:26.609550 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:26.609502    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6" podStartSLOduration=1.60948647 podStartE2EDuration="1.60948647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:17:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:17:26.60729338 +0000 UTC m=+1703.259484229" watchObservedRunningTime="2026-07-26 18:17:26.60948647 +0000 UTC m=+1703.261677328"
Jul 26 18:17:32.577576 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:32.577544    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr"
Jul 26 18:17:37.574581 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:37.574546    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6"
Jul 26 18:17:37.574978 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:37.574724    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m"
Jul 26 18:17:50.507286 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:50.507248    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a0608e5c-zx2sx"]
Jul 26 18:17:50.510960 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:50.510937    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a0608e5c-zx2sx"
Jul 26 18:17:50.513514 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:50.513485    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-swzk5\""
Jul 26 18:17:50.516874 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:50.516851    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a0608e5c-zx2sx"]
Jul 26 18:17:50.575552 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:50.575517    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e287dc5-35ce-42d8-b054-81b064bebdbe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a0608e5c-zx2sx\" (UID: \"5e287dc5-35ce-42d8-b054-81b064bebdbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a0608e5c-zx2sx"
Jul 26 18:17:50.676391 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:50.676354    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e287dc5-35ce-42d8-b054-81b064bebdbe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a0608e5c-zx2sx\" (UID: \"5e287dc5-35ce-42d8-b054-81b064bebdbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a0608e5c-zx2sx"
Jul 26 18:17:50.676994 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:50.676973    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e287dc5-35ce-42d8-b054-81b064bebdbe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a0608e5c-zx2sx\" (UID: \"5e287dc5-35ce-42d8-b054-81b064bebdbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a0608e5c-zx2sx"
Jul 26 18:17:50.823283 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:50.823250    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a0608e5c-zx2sx"
Jul 26 18:17:50.954910 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:50.954877    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a0608e5c-zx2sx"]
Jul 26 18:17:50.956499 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:17:50.956470    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e287dc5_35ce_42d8_b054_81b064bebdbe.slice/crio-d1e6e2e722d5e9a9c327c642133ccd0f11f33c1a14078e2fe5e644264c3bce25 WatchSource:0}: Error finding container d1e6e2e722d5e9a9c327c642133ccd0f11f33c1a14078e2fe5e644264c3bce25: Status 404 returned error can't find the container with id d1e6e2e722d5e9a9c327c642133ccd0f11f33c1a14078e2fe5e644264c3bce25
Jul 26 18:17:51.708345 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:51.708312    2570 generic.go:356] "Generic (PLEG): container finished" podID="5e287dc5-35ce-42d8-b054-81b064bebdbe" containerID="6855f7408ad9232a3eda0268308dccd87ace905f7064f56a96424c2ca246c2b4" exitCode=0
Jul 26 18:17:51.708730 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:51.708367    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a0608e5c-zx2sx" event={"ID":"5e287dc5-35ce-42d8-b054-81b064bebdbe","Type":"ContainerDied","Data":"6855f7408ad9232a3eda0268308dccd87ace905f7064f56a96424c2ca246c2b4"}
Jul 26 18:17:51.708730 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:51.708416    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a0608e5c-zx2sx" event={"ID":"5e287dc5-35ce-42d8-b054-81b064bebdbe","Type":"ContainerStarted","Data":"d1e6e2e722d5e9a9c327c642133ccd0f11f33c1a14078e2fe5e644264c3bce25"}
Jul 26 18:17:52.851356 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:52.851333    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a0608e5c-zx2sx"
Jul 26 18:17:52.896122 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:52.896073    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e287dc5-35ce-42d8-b054-81b064bebdbe-maas-api-service-ca\") pod \"5e287dc5-35ce-42d8-b054-81b064bebdbe\" (UID: \"5e287dc5-35ce-42d8-b054-81b064bebdbe\") "
Jul 26 18:17:52.896519 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:52.896490    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e287dc5-35ce-42d8-b054-81b064bebdbe-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5e287dc5-35ce-42d8-b054-81b064bebdbe" (UID: "5e287dc5-35ce-42d8-b054-81b064bebdbe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:17:52.997676 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:52.997579    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e287dc5-35ce-42d8-b054-81b064bebdbe-maas-api-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:17:53.720140 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:53.720075    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a0608e5c-zx2sx"
Jul 26 18:17:53.720140 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:53.720103    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a0608e5c-zx2sx" event={"ID":"5e287dc5-35ce-42d8-b054-81b064bebdbe","Type":"ContainerDied","Data":"d1e6e2e722d5e9a9c327c642133ccd0f11f33c1a14078e2fe5e644264c3bce25"}
Jul 26 18:17:53.720353 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:17:53.720172    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1e6e2e722d5e9a9c327c642133ccd0f11f33c1a14078e2fe5e644264c3bce25"
Jul 26 18:18:01.076336 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.076298    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr"]
Jul 26 18:18:01.076897 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.076543    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr" podUID="7ff7e4a9-d8f8-4778-b2c1-e318097c8e29" containerName="maas-api" containerID="cri-o://65c70b29b487473e718e3ddf08e0fa5db7fe8f1c534a029b04b7865bed619d99" gracePeriod=30
Jul 26 18:18:01.122707 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.122669    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6"]
Jul 26 18:18:01.122913 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.122893    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6" podUID="1f1fdc55-bc8d-467c-b139-18392dda93cd" containerName="payload-processing" containerID="cri-o://762c63168215e29307d396d8cfc472053ba88a4158d1d3fdcd3d55aced807b55" gracePeriod=30
Jul 26 18:18:01.128687 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.128644    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m"]
Jul 26 18:18:01.128901 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.128857    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m" podUID="ecdb2082-167c-4571-a9a9-f2427c814a66" containerName="payload-pre-processing" containerID="cri-o://d463f4babd914b777872c58ca98abe915bd75730fec5c0e4cc23e5ee80b94636" gracePeriod=30
Jul 26 18:18:01.399136 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.399087    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr"
Jul 26 18:18:01.416074 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.416047    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6"
Jul 26 18:18:01.419760 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.419736    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m"
Jul 26 18:18:01.470949 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.470910    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ff7e4a9-d8f8-4778-b2c1-e318097c8e29-maas-api-tls\") pod \"7ff7e4a9-d8f8-4778-b2c1-e318097c8e29\" (UID: \"7ff7e4a9-d8f8-4778-b2c1-e318097c8e29\") "
Jul 26 18:18:01.471141 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.470958    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dknhf\" (UniqueName: \"kubernetes.io/projected/1f1fdc55-bc8d-467c-b139-18392dda93cd-kube-api-access-dknhf\") pod \"1f1fdc55-bc8d-467c-b139-18392dda93cd\" (UID: \"1f1fdc55-bc8d-467c-b139-18392dda93cd\") "
Jul 26 18:18:01.471141 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.471013    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff7e4a9-d8f8-4778-b2c1-e318097c8e29-openshift-service-ca\") pod \"7ff7e4a9-d8f8-4778-b2c1-e318097c8e29\" (UID: \"7ff7e4a9-d8f8-4778-b2c1-e318097c8e29\") "
Jul 26 18:18:01.471141 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.471059    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6m9c8\" (UniqueName: \"kubernetes.io/projected/7ff7e4a9-d8f8-4778-b2c1-e318097c8e29-kube-api-access-6m9c8\") pod \"7ff7e4a9-d8f8-4778-b2c1-e318097c8e29\" (UID: \"7ff7e4a9-d8f8-4778-b2c1-e318097c8e29\") "
Jul 26 18:18:01.471267 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.471171    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f1fdc55-bc8d-467c-b139-18392dda93cd-plugins-config-volume\") pod \"1f1fdc55-bc8d-467c-b139-18392dda93cd\" (UID: \"1f1fdc55-bc8d-467c-b139-18392dda93cd\") "
Jul 26 18:18:01.471344 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.471327    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lsll\" (UniqueName: \"kubernetes.io/projected/ecdb2082-167c-4571-a9a9-f2427c814a66-kube-api-access-5lsll\") pod \"ecdb2082-167c-4571-a9a9-f2427c814a66\" (UID: \"ecdb2082-167c-4571-a9a9-f2427c814a66\") "
Jul 26 18:18:01.471387 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.471365    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecdb2082-167c-4571-a9a9-f2427c814a66-plugins-config-volume\") pod \"ecdb2082-167c-4571-a9a9-f2427c814a66\" (UID: \"ecdb2082-167c-4571-a9a9-f2427c814a66\") "
Jul 26 18:18:01.471460 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.471434    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ff7e4a9-d8f8-4778-b2c1-e318097c8e29-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7ff7e4a9-d8f8-4778-b2c1-e318097c8e29" (UID: "7ff7e4a9-d8f8-4778-b2c1-e318097c8e29"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:18:01.471521 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.471490    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f1fdc55-bc8d-467c-b139-18392dda93cd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1f1fdc55-bc8d-467c-b139-18392dda93cd" (UID: "1f1fdc55-bc8d-467c-b139-18392dda93cd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:18:01.471847 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.471824    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f1fdc55-bc8d-467c-b139-18392dda93cd-plugins-config-volume\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:18:01.471951 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.471840    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecdb2082-167c-4571-a9a9-f2427c814a66-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ecdb2082-167c-4571-a9a9-f2427c814a66" (UID: "ecdb2082-167c-4571-a9a9-f2427c814a66"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:18:01.471951 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.471851    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff7e4a9-d8f8-4778-b2c1-e318097c8e29-openshift-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:18:01.473306 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.473283    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f1fdc55-bc8d-467c-b139-18392dda93cd-kube-api-access-dknhf" (OuterVolumeSpecName: "kube-api-access-dknhf") pod "1f1fdc55-bc8d-467c-b139-18392dda93cd" (UID: "1f1fdc55-bc8d-467c-b139-18392dda93cd"). InnerVolumeSpecName "kube-api-access-dknhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:18:01.473394 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.473344    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecdb2082-167c-4571-a9a9-f2427c814a66-kube-api-access-5lsll" (OuterVolumeSpecName: "kube-api-access-5lsll") pod "ecdb2082-167c-4571-a9a9-f2427c814a66" (UID: "ecdb2082-167c-4571-a9a9-f2427c814a66"). InnerVolumeSpecName "kube-api-access-5lsll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:18:01.473798 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.473781    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ff7e4a9-d8f8-4778-b2c1-e318097c8e29-kube-api-access-6m9c8" (OuterVolumeSpecName: "kube-api-access-6m9c8") pod "7ff7e4a9-d8f8-4778-b2c1-e318097c8e29" (UID: "7ff7e4a9-d8f8-4778-b2c1-e318097c8e29"). InnerVolumeSpecName "kube-api-access-6m9c8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:18:01.473909 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.473895    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ff7e4a9-d8f8-4778-b2c1-e318097c8e29-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7ff7e4a9-d8f8-4778-b2c1-e318097c8e29" (UID: "7ff7e4a9-d8f8-4778-b2c1-e318097c8e29"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:18:01.572554 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.572502    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lsll\" (UniqueName: \"kubernetes.io/projected/ecdb2082-167c-4571-a9a9-f2427c814a66-kube-api-access-5lsll\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:18:01.572554 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.572549    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecdb2082-167c-4571-a9a9-f2427c814a66-plugins-config-volume\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:18:01.572554 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.572561    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ff7e4a9-d8f8-4778-b2c1-e318097c8e29-maas-api-tls\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:18:01.572554 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.572570    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dknhf\" (UniqueName: \"kubernetes.io/projected/1f1fdc55-bc8d-467c-b139-18392dda93cd-kube-api-access-dknhf\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:18:01.572835 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.572579    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6m9c8\" (UniqueName: \"kubernetes.io/projected/7ff7e4a9-d8f8-4778-b2c1-e318097c8e29-kube-api-access-6m9c8\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:18:01.762579 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.762540    2570 generic.go:356] "Generic (PLEG): container finished" podID="1f1fdc55-bc8d-467c-b139-18392dda93cd" containerID="762c63168215e29307d396d8cfc472053ba88a4158d1d3fdcd3d55aced807b55" exitCode=0
Jul 26 18:18:01.762755 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.762581    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6"
Jul 26 18:18:01.762755 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.762627    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6" event={"ID":"1f1fdc55-bc8d-467c-b139-18392dda93cd","Type":"ContainerDied","Data":"762c63168215e29307d396d8cfc472053ba88a4158d1d3fdcd3d55aced807b55"}
Jul 26 18:18:01.762755 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.762681    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6" event={"ID":"1f1fdc55-bc8d-467c-b139-18392dda93cd","Type":"ContainerDied","Data":"b7dae36034a5c8f2d7587d2c2832b1890091743cd69dc157974c7f5301e7209d"}
Jul 26 18:18:01.762755 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.762710    2570 scope.go:117] "RemoveContainer" containerID="762c63168215e29307d396d8cfc472053ba88a4158d1d3fdcd3d55aced807b55"
Jul 26 18:18:01.764086 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.764059    2570 generic.go:356] "Generic (PLEG): container finished" podID="ecdb2082-167c-4571-a9a9-f2427c814a66" containerID="d463f4babd914b777872c58ca98abe915bd75730fec5c0e4cc23e5ee80b94636" exitCode=0
Jul 26 18:18:01.764201 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.764096    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m"
Jul 26 18:18:01.764201 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.764147    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m" event={"ID":"ecdb2082-167c-4571-a9a9-f2427c814a66","Type":"ContainerDied","Data":"d463f4babd914b777872c58ca98abe915bd75730fec5c0e4cc23e5ee80b94636"}
Jul 26 18:18:01.764201 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.764193    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m" event={"ID":"ecdb2082-167c-4571-a9a9-f2427c814a66","Type":"ContainerDied","Data":"ef91837a8bed05fdb7d3b396ac6bfbd95621c9f39e9ea883bd1a4c455da8905c"}
Jul 26 18:18:01.765659 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.765634    2570 generic.go:356] "Generic (PLEG): container finished" podID="7ff7e4a9-d8f8-4778-b2c1-e318097c8e29" containerID="65c70b29b487473e718e3ddf08e0fa5db7fe8f1c534a029b04b7865bed619d99" exitCode=0
Jul 26 18:18:01.765754 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.765700    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr"
Jul 26 18:18:01.765754 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.765719    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr" event={"ID":"7ff7e4a9-d8f8-4778-b2c1-e318097c8e29","Type":"ContainerDied","Data":"65c70b29b487473e718e3ddf08e0fa5db7fe8f1c534a029b04b7865bed619d99"}
Jul 26 18:18:01.765843 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.765759    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr" event={"ID":"7ff7e4a9-d8f8-4778-b2c1-e318097c8e29","Type":"ContainerDied","Data":"b79a2627121dcfb840d952777d2e3667837024e0037dfecbaacbdbe07a8531d0"}
Jul 26 18:18:01.775243 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.775219    2570 scope.go:117] "RemoveContainer" containerID="762c63168215e29307d396d8cfc472053ba88a4158d1d3fdcd3d55aced807b55"
Jul 26 18:18:01.775572 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:18:01.775546    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"762c63168215e29307d396d8cfc472053ba88a4158d1d3fdcd3d55aced807b55\": container with ID starting with 762c63168215e29307d396d8cfc472053ba88a4158d1d3fdcd3d55aced807b55 not found: ID does not exist" containerID="762c63168215e29307d396d8cfc472053ba88a4158d1d3fdcd3d55aced807b55"
Jul 26 18:18:01.775650 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.775581    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"762c63168215e29307d396d8cfc472053ba88a4158d1d3fdcd3d55aced807b55"} err="failed to get container status \"762c63168215e29307d396d8cfc472053ba88a4158d1d3fdcd3d55aced807b55\": rpc error: code = NotFound desc = could not find container \"762c63168215e29307d396d8cfc472053ba88a4158d1d3fdcd3d55aced807b55\": container with ID starting with 762c63168215e29307d396d8cfc472053ba88a4158d1d3fdcd3d55aced807b55 not found: ID does not exist"
Jul 26 18:18:01.775650 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.775598    2570 scope.go:117] "RemoveContainer" containerID="d463f4babd914b777872c58ca98abe915bd75730fec5c0e4cc23e5ee80b94636"
Jul 26 18:18:01.784215 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.784194    2570 scope.go:117] "RemoveContainer" containerID="d463f4babd914b777872c58ca98abe915bd75730fec5c0e4cc23e5ee80b94636"
Jul 26 18:18:01.784504 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:18:01.784487    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d463f4babd914b777872c58ca98abe915bd75730fec5c0e4cc23e5ee80b94636\": container with ID starting with d463f4babd914b777872c58ca98abe915bd75730fec5c0e4cc23e5ee80b94636 not found: ID does not exist" containerID="d463f4babd914b777872c58ca98abe915bd75730fec5c0e4cc23e5ee80b94636"
Jul 26 18:18:01.784579 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.784510    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d463f4babd914b777872c58ca98abe915bd75730fec5c0e4cc23e5ee80b94636"} err="failed to get container status \"d463f4babd914b777872c58ca98abe915bd75730fec5c0e4cc23e5ee80b94636\": rpc error: code = NotFound desc = could not find container \"d463f4babd914b777872c58ca98abe915bd75730fec5c0e4cc23e5ee80b94636\": container with ID starting with d463f4babd914b777872c58ca98abe915bd75730fec5c0e4cc23e5ee80b94636 not found: ID does not exist"
Jul 26 18:18:01.784579 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.784527    2570 scope.go:117] "RemoveContainer" containerID="65c70b29b487473e718e3ddf08e0fa5db7fe8f1c534a029b04b7865bed619d99"
Jul 26 18:18:01.789736 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.789712    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6"]
Jul 26 18:18:01.792468 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.792442    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a0608e5c-55945964f6-lkhm6"]
Jul 26 18:18:01.796713 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.796688    2570 scope.go:117] "RemoveContainer" containerID="65c70b29b487473e718e3ddf08e0fa5db7fe8f1c534a029b04b7865bed619d99"
Jul 26 18:18:01.797312 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:18:01.797288    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65c70b29b487473e718e3ddf08e0fa5db7fe8f1c534a029b04b7865bed619d99\": container with ID starting with 65c70b29b487473e718e3ddf08e0fa5db7fe8f1c534a029b04b7865bed619d99 not found: ID does not exist" containerID="65c70b29b487473e718e3ddf08e0fa5db7fe8f1c534a029b04b7865bed619d99"
Jul 26 18:18:01.797396 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.797321    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65c70b29b487473e718e3ddf08e0fa5db7fe8f1c534a029b04b7865bed619d99"} err="failed to get container status \"65c70b29b487473e718e3ddf08e0fa5db7fe8f1c534a029b04b7865bed619d99\": rpc error: code = NotFound desc = could not find container \"65c70b29b487473e718e3ddf08e0fa5db7fe8f1c534a029b04b7865bed619d99\": container with ID starting with 65c70b29b487473e718e3ddf08e0fa5db7fe8f1c534a029b04b7865bed619d99 not found: ID does not exist"
Jul 26 18:18:01.805184 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.805153    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m"]
Jul 26 18:18:01.808709 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.808689    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a0608e5c-798bc67dcb-g4s6m"]
Jul 26 18:18:01.819857 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.819830    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr"]
Jul 26 18:18:01.823208 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.823186    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a0608e5c-796f4f8759-tf5hr"]
Jul 26 18:18:01.837859 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.837831    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f1fdc55-bc8d-467c-b139-18392dda93cd" path="/var/lib/kubelet/pods/1f1fdc55-bc8d-467c-b139-18392dda93cd/volumes"
Jul 26 18:18:01.838291 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.838276    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ff7e4a9-d8f8-4778-b2c1-e318097c8e29" path="/var/lib/kubelet/pods/7ff7e4a9-d8f8-4778-b2c1-e318097c8e29/volumes"
Jul 26 18:18:01.838625 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:01.838611    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecdb2082-167c-4571-a9a9-f2427c814a66" path="/var/lib/kubelet/pods/ecdb2082-167c-4571-a9a9-f2427c814a66/volumes"
Jul 26 18:18:05.596944 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:05.596895    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a0608e5c-zx2sx"]
Jul 26 18:18:05.598951 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:05.598928    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a0608e5c-zx2sx"]
Jul 26 18:18:05.838544 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:05.838509    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e287dc5-35ce-42d8-b054-81b064bebdbe" path="/var/lib/kubelet/pods/5e287dc5-35ce-42d8-b054-81b064bebdbe/volumes"
Jul 26 18:18:09.468914 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:09.468878    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw"]
Jul 26 18:18:09.469458 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:09.469441    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ff7e4a9-d8f8-4778-b2c1-e318097c8e29" containerName="maas-api"
Jul 26 18:18:09.469506 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:09.469461    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ff7e4a9-d8f8-4778-b2c1-e318097c8e29" containerName="maas-api"
Jul 26 18:18:09.469506 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:09.469488    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e287dc5-35ce-42d8-b054-81b064bebdbe" containerName="revoke-keys"
Jul 26 18:18:09.469506 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:09.469496    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e287dc5-35ce-42d8-b054-81b064bebdbe" containerName="revoke-keys"
Jul 26 18:18:09.469603 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:09.469508    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f1fdc55-bc8d-467c-b139-18392dda93cd" containerName="payload-processing"
Jul 26 18:18:09.469603 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:09.469517    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f1fdc55-bc8d-467c-b139-18392dda93cd" containerName="payload-processing"
Jul 26 18:18:09.469603 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:09.469534    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecdb2082-167c-4571-a9a9-f2427c814a66" containerName="payload-pre-processing"
Jul 26 18:18:09.469603 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:09.469543    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecdb2082-167c-4571-a9a9-f2427c814a66" containerName="payload-pre-processing"
Jul 26 18:18:09.469728 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:09.469632    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecdb2082-167c-4571-a9a9-f2427c814a66" containerName="payload-pre-processing"
Jul 26 18:18:09.469728 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:09.469645    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e287dc5-35ce-42d8-b054-81b064bebdbe" containerName="revoke-keys"
Jul 26 18:18:09.469728 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:09.469657    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f1fdc55-bc8d-467c-b139-18392dda93cd" containerName="payload-processing"
Jul 26 18:18:09.469728 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:09.469668    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ff7e4a9-d8f8-4778-b2c1-e318097c8e29" containerName="maas-api"
Jul 26 18:18:09.473250 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:09.473228    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw"
Jul 26 18:18:09.475624 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:09.475600    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-47ecefc5-serving-cert\""
Jul 26 18:18:09.483347 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:09.483317    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw"]
Jul 26 18:18:09.539785 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:09.539744    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/955d96da-38a9-4f3c-8b3c-ec53ba49cfe5-maas-api-tls\") pod \"maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw\" (UID: \"955d96da-38a9-4f3c-8b3c-ec53ba49cfe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw"
Jul 26 18:18:09.539979 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:09.539816    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/955d96da-38a9-4f3c-8b3c-ec53ba49cfe5-openshift-service-ca\") pod \"maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw\" (UID: \"955d96da-38a9-4f3c-8b3c-ec53ba49cfe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw"
Jul 26 18:18:09.539979 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:09.539842    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh7rq\" (UniqueName: \"kubernetes.io/projected/955d96da-38a9-4f3c-8b3c-ec53ba49cfe5-kube-api-access-kh7rq\") pod \"maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw\" (UID: \"955d96da-38a9-4f3c-8b3c-ec53ba49cfe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw"
Jul 26 18:18:09.640419 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:09.640380    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/955d96da-38a9-4f3c-8b3c-ec53ba49cfe5-maas-api-tls\") pod \"maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw\" (UID: \"955d96da-38a9-4f3c-8b3c-ec53ba49cfe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw"
Jul 26 18:18:09.640570 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:09.640449    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/955d96da-38a9-4f3c-8b3c-ec53ba49cfe5-openshift-service-ca\") pod \"maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw\" (UID: \"955d96da-38a9-4f3c-8b3c-ec53ba49cfe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw"
Jul 26 18:18:09.640570 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:09.640469    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh7rq\" (UniqueName: \"kubernetes.io/projected/955d96da-38a9-4f3c-8b3c-ec53ba49cfe5-kube-api-access-kh7rq\") pod \"maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw\" (UID: \"955d96da-38a9-4f3c-8b3c-ec53ba49cfe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw"
Jul 26 18:18:09.640570 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:18:09.640526    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-serving-cert: secret "maas-api-e2e-migrate-47ecefc5-serving-cert" not found
Jul 26 18:18:09.640700 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:18:09.640594    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/955d96da-38a9-4f3c-8b3c-ec53ba49cfe5-maas-api-tls podName:955d96da-38a9-4f3c-8b3c-ec53ba49cfe5 nodeName:}" failed. No retries permitted until 2026-07-26 18:18:10.140572426 +0000 UTC m=+1746.792763262 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/955d96da-38a9-4f3c-8b3c-ec53ba49cfe5-maas-api-tls") pod "maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw" (UID: "955d96da-38a9-4f3c-8b3c-ec53ba49cfe5") : secret "maas-api-e2e-migrate-47ecefc5-serving-cert" not found
Jul 26 18:18:09.641073 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:09.641053    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/955d96da-38a9-4f3c-8b3c-ec53ba49cfe5-openshift-service-ca\") pod \"maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw\" (UID: \"955d96da-38a9-4f3c-8b3c-ec53ba49cfe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw"
Jul 26 18:18:09.651297 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:09.651269    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh7rq\" (UniqueName: \"kubernetes.io/projected/955d96da-38a9-4f3c-8b3c-ec53ba49cfe5-kube-api-access-kh7rq\") pod \"maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw\" (UID: \"955d96da-38a9-4f3c-8b3c-ec53ba49cfe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw"
Jul 26 18:18:10.108333 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.108297    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr"]
Jul 26 18:18:10.111814 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.111797    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr"
Jul 26 18:18:10.114220 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.114197    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-47ecefc5\""
Jul 26 18:18:10.114331 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.114197    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-47ecefc5-dockercfg-7hgqn\""
Jul 26 18:18:10.124102 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.124078    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr"]
Jul 26 18:18:10.141826 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.141797    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g"]
Jul 26 18:18:10.145085 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.145062    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g"
Jul 26 18:18:10.145469 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.145447    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/955d96da-38a9-4f3c-8b3c-ec53ba49cfe5-maas-api-tls\") pod \"maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw\" (UID: \"955d96da-38a9-4f3c-8b3c-ec53ba49cfe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw"
Jul 26 18:18:10.147754 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.147736    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/955d96da-38a9-4f3c-8b3c-ec53ba49cfe5-maas-api-tls\") pod \"maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw\" (UID: \"955d96da-38a9-4f3c-8b3c-ec53ba49cfe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw"
Jul 26 18:18:10.155626 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.155600    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g"]
Jul 26 18:18:10.246445 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.246398    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2846139f-4b63-4145-90c3-62f1dedf76f5-plugins-config-volume\") pod \"payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g\" (UID: \"2846139f-4b63-4145-90c3-62f1dedf76f5\") " pod="openshift-ingress/payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g"
Jul 26 18:18:10.246445 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.246442    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcc7h\" (UniqueName: \"kubernetes.io/projected/2846139f-4b63-4145-90c3-62f1dedf76f5-kube-api-access-bcc7h\") pod \"payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g\" (UID: \"2846139f-4b63-4145-90c3-62f1dedf76f5\") " pod="openshift-ingress/payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g"
Jul 26 18:18:10.246658 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.246527    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww7hb\" (UniqueName: \"kubernetes.io/projected/2b232078-9d40-41fc-a047-215b43bd3a83-kube-api-access-ww7hb\") pod \"payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr\" (UID: \"2b232078-9d40-41fc-a047-215b43bd3a83\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr"
Jul 26 18:18:10.246658 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.246576    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b232078-9d40-41fc-a047-215b43bd3a83-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr\" (UID: \"2b232078-9d40-41fc-a047-215b43bd3a83\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr"
Jul 26 18:18:10.347286 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.347240    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww7hb\" (UniqueName: \"kubernetes.io/projected/2b232078-9d40-41fc-a047-215b43bd3a83-kube-api-access-ww7hb\") pod \"payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr\" (UID: \"2b232078-9d40-41fc-a047-215b43bd3a83\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr"
Jul 26 18:18:10.347286 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.347290    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b232078-9d40-41fc-a047-215b43bd3a83-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr\" (UID: \"2b232078-9d40-41fc-a047-215b43bd3a83\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr"
Jul 26 18:18:10.347552 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.347338    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2846139f-4b63-4145-90c3-62f1dedf76f5-plugins-config-volume\") pod \"payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g\" (UID: \"2846139f-4b63-4145-90c3-62f1dedf76f5\") " pod="openshift-ingress/payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g"
Jul 26 18:18:10.347552 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.347355    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcc7h\" (UniqueName: \"kubernetes.io/projected/2846139f-4b63-4145-90c3-62f1dedf76f5-kube-api-access-bcc7h\") pod \"payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g\" (UID: \"2846139f-4b63-4145-90c3-62f1dedf76f5\") " pod="openshift-ingress/payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g"
Jul 26 18:18:10.348205 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.348096    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b232078-9d40-41fc-a047-215b43bd3a83-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr\" (UID: \"2b232078-9d40-41fc-a047-215b43bd3a83\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr"
Jul 26 18:18:10.348205 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.348150    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2846139f-4b63-4145-90c3-62f1dedf76f5-plugins-config-volume\") pod \"payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g\" (UID: \"2846139f-4b63-4145-90c3-62f1dedf76f5\") " pod="openshift-ingress/payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g"
Jul 26 18:18:10.355671 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.355647    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww7hb\" (UniqueName: \"kubernetes.io/projected/2b232078-9d40-41fc-a047-215b43bd3a83-kube-api-access-ww7hb\") pod \"payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr\" (UID: \"2b232078-9d40-41fc-a047-215b43bd3a83\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr"
Jul 26 18:18:10.355774 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.355706    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcc7h\" (UniqueName: \"kubernetes.io/projected/2846139f-4b63-4145-90c3-62f1dedf76f5-kube-api-access-bcc7h\") pod \"payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g\" (UID: \"2846139f-4b63-4145-90c3-62f1dedf76f5\") " pod="openshift-ingress/payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g"
Jul 26 18:18:10.384577 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.384488    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw"
Jul 26 18:18:10.422674 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.422632    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr"
Jul 26 18:18:10.472148 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.471842    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g"
Jul 26 18:18:10.532679 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:18:10.532645    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod955d96da_38a9_4f3c_8b3c_ec53ba49cfe5.slice/crio-752606c53b840fa53dc50cb594384da56d3c05fafa89da46bd497909a69cc1a4 WatchSource:0}: Error finding container 752606c53b840fa53dc50cb594384da56d3c05fafa89da46bd497909a69cc1a4: Status 404 returned error can't find the container with id 752606c53b840fa53dc50cb594384da56d3c05fafa89da46bd497909a69cc1a4
Jul 26 18:18:10.532919 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.532892    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw"]
Jul 26 18:18:10.586869 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.586809    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr"]
Jul 26 18:18:10.587982 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:18:10.587944    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b232078_9d40_41fc_a047_215b43bd3a83.slice/crio-fc1ecfe5c93cc987ea47784fd4b9155edbb6d87f306bb4b3c498093e58935c59 WatchSource:0}: Error finding container fc1ecfe5c93cc987ea47784fd4b9155edbb6d87f306bb4b3c498093e58935c59: Status 404 returned error can't find the container with id fc1ecfe5c93cc987ea47784fd4b9155edbb6d87f306bb4b3c498093e58935c59
Jul 26 18:18:10.632230 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.632206    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g"]
Jul 26 18:18:10.633901 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:18:10.633876    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2846139f_4b63_4145_90c3_62f1dedf76f5.slice/crio-80860b3e0d6b4cd98ab1e9202a0c4fa827196e5d167346cd19d9ff165e0a5b60 WatchSource:0}: Error finding container 80860b3e0d6b4cd98ab1e9202a0c4fa827196e5d167346cd19d9ff165e0a5b60: Status 404 returned error can't find the container with id 80860b3e0d6b4cd98ab1e9202a0c4fa827196e5d167346cd19d9ff165e0a5b60
Jul 26 18:18:10.827590 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.827544    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr" event={"ID":"2b232078-9d40-41fc-a047-215b43bd3a83","Type":"ContainerStarted","Data":"1d10f5aff0298ff77b28c35152ea26964bed7ab74b601ee99ec7a2b9103a0764"}
Jul 26 18:18:10.827756 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.827607    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr" event={"ID":"2b232078-9d40-41fc-a047-215b43bd3a83","Type":"ContainerStarted","Data":"fc1ecfe5c93cc987ea47784fd4b9155edbb6d87f306bb4b3c498093e58935c59"}
Jul 26 18:18:10.827756 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.827631    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr"
Jul 26 18:18:10.829142 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.829088    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g" event={"ID":"2846139f-4b63-4145-90c3-62f1dedf76f5","Type":"ContainerStarted","Data":"d07bf23ceb438c05b883d8ec9399f46f6f0ded7dffbbfc283c3bc1cc813b4b86"}
Jul 26 18:18:10.829279 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.829164    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g" event={"ID":"2846139f-4b63-4145-90c3-62f1dedf76f5","Type":"ContainerStarted","Data":"80860b3e0d6b4cd98ab1e9202a0c4fa827196e5d167346cd19d9ff165e0a5b60"}
Jul 26 18:18:10.829279 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.829193    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g"
Jul 26 18:18:10.830404 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.830382    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw" event={"ID":"955d96da-38a9-4f3c-8b3c-ec53ba49cfe5","Type":"ContainerStarted","Data":"752606c53b840fa53dc50cb594384da56d3c05fafa89da46bd497909a69cc1a4"}
Jul 26 18:18:10.844214 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.844168    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr" podStartSLOduration=0.844147941 podStartE2EDuration="844.147941ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:18:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:18:10.841053366 +0000 UTC m=+1747.493244237" watchObservedRunningTime="2026-07-26 18:18:10.844147941 +0000 UTC m=+1747.496338797"
Jul 26 18:18:10.854909 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:10.854855    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g" podStartSLOduration=0.854838106 podStartE2EDuration="854.838106ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:18:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:18:10.85445568 +0000 UTC m=+1747.506646539" watchObservedRunningTime="2026-07-26 18:18:10.854838106 +0000 UTC m=+1747.507028965"
Jul 26 18:18:11.848570 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:11.848536    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw" event={"ID":"955d96da-38a9-4f3c-8b3c-ec53ba49cfe5","Type":"ContainerStarted","Data":"e558cb11bbf5493643c3b43f09380fe8a5433e9df8e158a4ac6e623f59546fbd"}
Jul 26 18:18:11.863892 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:11.863839    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw" podStartSLOduration=2.468475697 podStartE2EDuration="2.863821842s" totalImagesPullingTime="395.346145ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:18:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:18:11.862364616 +0000 UTC m=+1748.514555477" watchObservedRunningTime="2026-07-26 18:18:11.863821842 +0000 UTC m=+1748.516012701"
Jul 26 18:18:12.853192 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:12.853157    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw"
Jul 26 18:18:18.863795 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:18.863762    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw"
Jul 26 18:18:21.850199 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:21.850158    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g"
Jul 26 18:18:21.850641 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:21.850246    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr"
Jul 26 18:18:34.373398 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:34.373367    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ecefc5-c2xdt"]
Jul 26 18:18:34.378417 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:34.378402    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ecefc5-c2xdt"
Jul 26 18:18:34.381138 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:34.380871    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-swzk5\""
Jul 26 18:18:34.384569 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:34.384540    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ecefc5-c2xdt"]
Jul 26 18:18:34.479632 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:34.479591    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf79eb08-291a-4a49-b437-b2989c5e1a8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-47ecefc5-c2xdt\" (UID: \"cf79eb08-291a-4a49-b437-b2989c5e1a8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ecefc5-c2xdt"
Jul 26 18:18:34.580762 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:34.580726    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf79eb08-291a-4a49-b437-b2989c5e1a8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-47ecefc5-c2xdt\" (UID: \"cf79eb08-291a-4a49-b437-b2989c5e1a8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ecefc5-c2xdt"
Jul 26 18:18:34.581465 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:34.581438    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf79eb08-291a-4a49-b437-b2989c5e1a8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-47ecefc5-c2xdt\" (UID: \"cf79eb08-291a-4a49-b437-b2989c5e1a8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ecefc5-c2xdt"
Jul 26 18:18:34.691245 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:34.691167    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ecefc5-c2xdt"
Jul 26 18:18:35.029422 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:35.027029    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ecefc5-c2xdt"]
Jul 26 18:18:35.029990 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:18:35.029959    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf79eb08_291a_4a49_b437_b2989c5e1a8d.slice/crio-88fac0582bdede2696196f0b9c6be6514882c523780a3cfdbb9c28969d84a7d8 WatchSource:0}: Error finding container 88fac0582bdede2696196f0b9c6be6514882c523780a3cfdbb9c28969d84a7d8: Status 404 returned error can't find the container with id 88fac0582bdede2696196f0b9c6be6514882c523780a3cfdbb9c28969d84a7d8
Jul 26 18:18:35.980476 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:35.980437    2570 generic.go:356] "Generic (PLEG): container finished" podID="cf79eb08-291a-4a49-b437-b2989c5e1a8d" containerID="b665600242cfa969495e925bd6c7e29eb8dc48aa395463c227261693dc57a3d0" exitCode=0
Jul 26 18:18:35.980859 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:35.980531    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ecefc5-c2xdt" event={"ID":"cf79eb08-291a-4a49-b437-b2989c5e1a8d","Type":"ContainerDied","Data":"b665600242cfa969495e925bd6c7e29eb8dc48aa395463c227261693dc57a3d0"}
Jul 26 18:18:35.980859 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:35.980589    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ecefc5-c2xdt" event={"ID":"cf79eb08-291a-4a49-b437-b2989c5e1a8d","Type":"ContainerStarted","Data":"88fac0582bdede2696196f0b9c6be6514882c523780a3cfdbb9c28969d84a7d8"}
Jul 26 18:18:37.117564 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:37.117536    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ecefc5-c2xdt"
Jul 26 18:18:37.203394 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:37.203355    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf79eb08-291a-4a49-b437-b2989c5e1a8d-maas-api-service-ca\") pod \"cf79eb08-291a-4a49-b437-b2989c5e1a8d\" (UID: \"cf79eb08-291a-4a49-b437-b2989c5e1a8d\") "
Jul 26 18:18:37.203765 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:37.203744    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf79eb08-291a-4a49-b437-b2989c5e1a8d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cf79eb08-291a-4a49-b437-b2989c5e1a8d" (UID: "cf79eb08-291a-4a49-b437-b2989c5e1a8d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:18:37.304581 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:37.304543    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf79eb08-291a-4a49-b437-b2989c5e1a8d-maas-api-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:18:37.992175 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:37.992141    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ecefc5-c2xdt" event={"ID":"cf79eb08-291a-4a49-b437-b2989c5e1a8d","Type":"ContainerDied","Data":"88fac0582bdede2696196f0b9c6be6514882c523780a3cfdbb9c28969d84a7d8"}
Jul 26 18:18:37.992358 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:37.992204    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="88fac0582bdede2696196f0b9c6be6514882c523780a3cfdbb9c28969d84a7d8"
Jul 26 18:18:37.992358 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:37.992149    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ecefc5-c2xdt"
Jul 26 18:18:44.809006 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:44.808969    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw"]
Jul 26 18:18:44.809496 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:44.809219    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw" podUID="955d96da-38a9-4f3c-8b3c-ec53ba49cfe5" containerName="maas-api" containerID="cri-o://e558cb11bbf5493643c3b43f09380fe8a5433e9df8e158a4ac6e623f59546fbd" gracePeriod=30
Jul 26 18:18:44.861031 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:44.860998    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g"]
Jul 26 18:18:44.861266 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:44.861240    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g" podUID="2846139f-4b63-4145-90c3-62f1dedf76f5" containerName="payload-processing" containerID="cri-o://d07bf23ceb438c05b883d8ec9399f46f6f0ded7dffbbfc283c3bc1cc813b4b86" gracePeriod=30
Jul 26 18:18:44.869994 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:44.869964    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr"]
Jul 26 18:18:44.870213 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:44.870193    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr" podUID="2b232078-9d40-41fc-a047-215b43bd3a83" containerName="payload-pre-processing" containerID="cri-o://1d10f5aff0298ff77b28c35152ea26964bed7ab74b601ee99ec7a2b9103a0764" gracePeriod=30
Jul 26 18:18:45.032726 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.032688    2570 generic.go:356] "Generic (PLEG): container finished" podID="2846139f-4b63-4145-90c3-62f1dedf76f5" containerID="d07bf23ceb438c05b883d8ec9399f46f6f0ded7dffbbfc283c3bc1cc813b4b86" exitCode=0
Jul 26 18:18:45.032934 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.032762    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g" event={"ID":"2846139f-4b63-4145-90c3-62f1dedf76f5","Type":"ContainerDied","Data":"d07bf23ceb438c05b883d8ec9399f46f6f0ded7dffbbfc283c3bc1cc813b4b86"}
Jul 26 18:18:45.034483 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.034459    2570 generic.go:356] "Generic (PLEG): container finished" podID="955d96da-38a9-4f3c-8b3c-ec53ba49cfe5" containerID="e558cb11bbf5493643c3b43f09380fe8a5433e9df8e158a4ac6e623f59546fbd" exitCode=0
Jul 26 18:18:45.034726 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.034518    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw" event={"ID":"955d96da-38a9-4f3c-8b3c-ec53ba49cfe5","Type":"ContainerDied","Data":"e558cb11bbf5493643c3b43f09380fe8a5433e9df8e158a4ac6e623f59546fbd"}
Jul 26 18:18:45.041830 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.037126    2570 generic.go:356] "Generic (PLEG): container finished" podID="2b232078-9d40-41fc-a047-215b43bd3a83" containerID="1d10f5aff0298ff77b28c35152ea26964bed7ab74b601ee99ec7a2b9103a0764" exitCode=0
Jul 26 18:18:45.041830 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.037186    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr" event={"ID":"2b232078-9d40-41fc-a047-215b43bd3a83","Type":"ContainerDied","Data":"1d10f5aff0298ff77b28c35152ea26964bed7ab74b601ee99ec7a2b9103a0764"}
Jul 26 18:18:45.150300 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.150273    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g"
Jul 26 18:18:45.187147 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.187105    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw"
Jul 26 18:18:45.190557 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.190533    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr"
Jul 26 18:18:45.272292 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.272253    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/955d96da-38a9-4f3c-8b3c-ec53ba49cfe5-openshift-service-ca\") pod \"955d96da-38a9-4f3c-8b3c-ec53ba49cfe5\" (UID: \"955d96da-38a9-4f3c-8b3c-ec53ba49cfe5\") "
Jul 26 18:18:45.272292 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.272289    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kh7rq\" (UniqueName: \"kubernetes.io/projected/955d96da-38a9-4f3c-8b3c-ec53ba49cfe5-kube-api-access-kh7rq\") pod \"955d96da-38a9-4f3c-8b3c-ec53ba49cfe5\" (UID: \"955d96da-38a9-4f3c-8b3c-ec53ba49cfe5\") "
Jul 26 18:18:45.272524 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.272347    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcc7h\" (UniqueName: \"kubernetes.io/projected/2846139f-4b63-4145-90c3-62f1dedf76f5-kube-api-access-bcc7h\") pod \"2846139f-4b63-4145-90c3-62f1dedf76f5\" (UID: \"2846139f-4b63-4145-90c3-62f1dedf76f5\") "
Jul 26 18:18:45.272524 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.272387    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/955d96da-38a9-4f3c-8b3c-ec53ba49cfe5-maas-api-tls\") pod \"955d96da-38a9-4f3c-8b3c-ec53ba49cfe5\" (UID: \"955d96da-38a9-4f3c-8b3c-ec53ba49cfe5\") "
Jul 26 18:18:45.272524 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.272427    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2846139f-4b63-4145-90c3-62f1dedf76f5-plugins-config-volume\") pod \"2846139f-4b63-4145-90c3-62f1dedf76f5\" (UID: \"2846139f-4b63-4145-90c3-62f1dedf76f5\") "
Jul 26 18:18:45.272524 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.272479    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b232078-9d40-41fc-a047-215b43bd3a83-plugins-config-volume\") pod \"2b232078-9d40-41fc-a047-215b43bd3a83\" (UID: \"2b232078-9d40-41fc-a047-215b43bd3a83\") "
Jul 26 18:18:45.272524 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.272512    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ww7hb\" (UniqueName: \"kubernetes.io/projected/2b232078-9d40-41fc-a047-215b43bd3a83-kube-api-access-ww7hb\") pod \"2b232078-9d40-41fc-a047-215b43bd3a83\" (UID: \"2b232078-9d40-41fc-a047-215b43bd3a83\") "
Jul 26 18:18:45.272858 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.272709    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/955d96da-38a9-4f3c-8b3c-ec53ba49cfe5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "955d96da-38a9-4f3c-8b3c-ec53ba49cfe5" (UID: "955d96da-38a9-4f3c-8b3c-ec53ba49cfe5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:18:45.272858 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.272795    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/955d96da-38a9-4f3c-8b3c-ec53ba49cfe5-openshift-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:18:45.272964 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.272877    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b232078-9d40-41fc-a047-215b43bd3a83-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2b232078-9d40-41fc-a047-215b43bd3a83" (UID: "2b232078-9d40-41fc-a047-215b43bd3a83"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:18:45.272964 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.272882    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2846139f-4b63-4145-90c3-62f1dedf76f5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2846139f-4b63-4145-90c3-62f1dedf76f5" (UID: "2846139f-4b63-4145-90c3-62f1dedf76f5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:18:45.274695 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.274657    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2846139f-4b63-4145-90c3-62f1dedf76f5-kube-api-access-bcc7h" (OuterVolumeSpecName: "kube-api-access-bcc7h") pod "2846139f-4b63-4145-90c3-62f1dedf76f5" (UID: "2846139f-4b63-4145-90c3-62f1dedf76f5"). InnerVolumeSpecName "kube-api-access-bcc7h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:18:45.274796 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.274708    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b232078-9d40-41fc-a047-215b43bd3a83-kube-api-access-ww7hb" (OuterVolumeSpecName: "kube-api-access-ww7hb") pod "2b232078-9d40-41fc-a047-215b43bd3a83" (UID: "2b232078-9d40-41fc-a047-215b43bd3a83"). InnerVolumeSpecName "kube-api-access-ww7hb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:18:45.274908 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.274890    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/955d96da-38a9-4f3c-8b3c-ec53ba49cfe5-kube-api-access-kh7rq" (OuterVolumeSpecName: "kube-api-access-kh7rq") pod "955d96da-38a9-4f3c-8b3c-ec53ba49cfe5" (UID: "955d96da-38a9-4f3c-8b3c-ec53ba49cfe5"). InnerVolumeSpecName "kube-api-access-kh7rq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:18:45.274951 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.274914    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/955d96da-38a9-4f3c-8b3c-ec53ba49cfe5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "955d96da-38a9-4f3c-8b3c-ec53ba49cfe5" (UID: "955d96da-38a9-4f3c-8b3c-ec53ba49cfe5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:18:45.373733 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.373626    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kh7rq\" (UniqueName: \"kubernetes.io/projected/955d96da-38a9-4f3c-8b3c-ec53ba49cfe5-kube-api-access-kh7rq\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:18:45.373733 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.373672    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcc7h\" (UniqueName: \"kubernetes.io/projected/2846139f-4b63-4145-90c3-62f1dedf76f5-kube-api-access-bcc7h\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:18:45.373733 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.373682    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/955d96da-38a9-4f3c-8b3c-ec53ba49cfe5-maas-api-tls\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:18:45.373733 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.373692    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2846139f-4b63-4145-90c3-62f1dedf76f5-plugins-config-volume\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:18:45.373733 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.373701    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b232078-9d40-41fc-a047-215b43bd3a83-plugins-config-volume\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:18:45.373733 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:45.373709    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ww7hb\" (UniqueName: \"kubernetes.io/projected/2b232078-9d40-41fc-a047-215b43bd3a83-kube-api-access-ww7hb\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:18:46.044317 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:46.044279    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g"
Jul 26 18:18:46.044763 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:46.044272    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g" event={"ID":"2846139f-4b63-4145-90c3-62f1dedf76f5","Type":"ContainerDied","Data":"80860b3e0d6b4cd98ab1e9202a0c4fa827196e5d167346cd19d9ff165e0a5b60"}
Jul 26 18:18:46.044763 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:46.044420    2570 scope.go:117] "RemoveContainer" containerID="d07bf23ceb438c05b883d8ec9399f46f6f0ded7dffbbfc283c3bc1cc813b4b86"
Jul 26 18:18:46.045780 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:46.045617    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw"
Jul 26 18:18:46.045780 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:46.045703    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw" event={"ID":"955d96da-38a9-4f3c-8b3c-ec53ba49cfe5","Type":"ContainerDied","Data":"752606c53b840fa53dc50cb594384da56d3c05fafa89da46bd497909a69cc1a4"}
Jul 26 18:18:46.047337 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:46.047315    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr" event={"ID":"2b232078-9d40-41fc-a047-215b43bd3a83","Type":"ContainerDied","Data":"fc1ecfe5c93cc987ea47784fd4b9155edbb6d87f306bb4b3c498093e58935c59"}
Jul 26 18:18:46.047430 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:46.047346    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr"
Jul 26 18:18:46.053955 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:46.053940    2570 scope.go:117] "RemoveContainer" containerID="e558cb11bbf5493643c3b43f09380fe8a5433e9df8e158a4ac6e623f59546fbd"
Jul 26 18:18:46.061763 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:46.061748    2570 scope.go:117] "RemoveContainer" containerID="1d10f5aff0298ff77b28c35152ea26964bed7ab74b601ee99ec7a2b9103a0764"
Jul 26 18:18:46.065177 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:46.065155    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw"]
Jul 26 18:18:46.068980 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:46.068960    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-47ecefc5-7fc8649c8d-mvphw"]
Jul 26 18:18:46.080266 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:46.080211    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr"]
Jul 26 18:18:46.083911 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:46.083892    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-47ecefc5-78956f4d68-7j2jr"]
Jul 26 18:18:46.093606 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:46.093586    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g"]
Jul 26 18:18:46.097315 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:46.097290    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-47ecefc5-5bfb9cf489-8fm4g"]
Jul 26 18:18:47.841475 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:47.841443    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2846139f-4b63-4145-90c3-62f1dedf76f5" path="/var/lib/kubelet/pods/2846139f-4b63-4145-90c3-62f1dedf76f5/volumes"
Jul 26 18:18:47.841842 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:47.841784    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b232078-9d40-41fc-a047-215b43bd3a83" path="/var/lib/kubelet/pods/2b232078-9d40-41fc-a047-215b43bd3a83/volumes"
Jul 26 18:18:47.842175 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:47.842160    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="955d96da-38a9-4f3c-8b3c-ec53ba49cfe5" path="/var/lib/kubelet/pods/955d96da-38a9-4f3c-8b3c-ec53ba49cfe5/volumes"
Jul 26 18:18:49.453806 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:49.453759    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ecefc5-c2xdt"]
Jul 26 18:18:49.455883 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:49.455858    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ecefc5-c2xdt"]
Jul 26 18:18:49.837934 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:49.837899    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf79eb08-291a-4a49-b437-b2989c5e1a8d" path="/var/lib/kubelet/pods/cf79eb08-291a-4a49-b437-b2989c5e1a8d/volumes"
Jul 26 18:18:52.691183 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:52.691144    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr"]
Jul 26 18:18:52.691556 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:52.691479    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b232078-9d40-41fc-a047-215b43bd3a83" containerName="payload-pre-processing"
Jul 26 18:18:52.691556 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:52.691489    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b232078-9d40-41fc-a047-215b43bd3a83" containerName="payload-pre-processing"
Jul 26 18:18:52.691556 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:52.691498    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="955d96da-38a9-4f3c-8b3c-ec53ba49cfe5" containerName="maas-api"
Jul 26 18:18:52.691556 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:52.691505    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="955d96da-38a9-4f3c-8b3c-ec53ba49cfe5" containerName="maas-api"
Jul 26 18:18:52.691556 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:52.691513    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf79eb08-291a-4a49-b437-b2989c5e1a8d" containerName="revoke-keys"
Jul 26 18:18:52.691556 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:52.691519    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf79eb08-291a-4a49-b437-b2989c5e1a8d" containerName="revoke-keys"
Jul 26 18:18:52.691556 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:52.691538    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2846139f-4b63-4145-90c3-62f1dedf76f5" containerName="payload-processing"
Jul 26 18:18:52.691556 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:52.691543    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2846139f-4b63-4145-90c3-62f1dedf76f5" containerName="payload-processing"
Jul 26 18:18:52.691800 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:52.691598    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="955d96da-38a9-4f3c-8b3c-ec53ba49cfe5" containerName="maas-api"
Jul 26 18:18:52.691800 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:52.691605    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b232078-9d40-41fc-a047-215b43bd3a83" containerName="payload-pre-processing"
Jul 26 18:18:52.691800 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:52.691615    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf79eb08-291a-4a49-b437-b2989c5e1a8d" containerName="revoke-keys"
Jul 26 18:18:52.691800 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:52.691620    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2846139f-4b63-4145-90c3-62f1dedf76f5" containerName="payload-processing"
Jul 26 18:18:52.696225 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:52.696200    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr"
Jul 26 18:18:52.698816 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:52.698790    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-10cd5747-serving-cert\""
Jul 26 18:18:52.719647 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:52.719620    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr"]
Jul 26 18:18:52.840627 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:52.840589    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt2cl\" (UniqueName: \"kubernetes.io/projected/a2fca19c-eb56-419d-beca-6d9cd9f6e62f-kube-api-access-rt2cl\") pod \"maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr\" (UID: \"a2fca19c-eb56-419d-beca-6d9cd9f6e62f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr"
Jul 26 18:18:52.840773 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:52.840646    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2fca19c-eb56-419d-beca-6d9cd9f6e62f-openshift-service-ca\") pod \"maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr\" (UID: \"a2fca19c-eb56-419d-beca-6d9cd9f6e62f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr"
Jul 26 18:18:52.840773 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:52.840668    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2fca19c-eb56-419d-beca-6d9cd9f6e62f-maas-api-tls\") pod \"maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr\" (UID: \"a2fca19c-eb56-419d-beca-6d9cd9f6e62f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr"
Jul 26 18:18:52.941693 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:52.941604    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2fca19c-eb56-419d-beca-6d9cd9f6e62f-openshift-service-ca\") pod \"maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr\" (UID: \"a2fca19c-eb56-419d-beca-6d9cd9f6e62f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr"
Jul 26 18:18:52.941693 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:52.941642    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2fca19c-eb56-419d-beca-6d9cd9f6e62f-maas-api-tls\") pod \"maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr\" (UID: \"a2fca19c-eb56-419d-beca-6d9cd9f6e62f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr"
Jul 26 18:18:52.941693 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:52.941689    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt2cl\" (UniqueName: \"kubernetes.io/projected/a2fca19c-eb56-419d-beca-6d9cd9f6e62f-kube-api-access-rt2cl\") pod \"maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr\" (UID: \"a2fca19c-eb56-419d-beca-6d9cd9f6e62f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr"
Jul 26 18:18:52.941931 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:18:52.941816    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-serving-cert: secret "maas-api-e2e-ait-10cd5747-serving-cert" not found
Jul 26 18:18:52.941931 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:18:52.941897    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2fca19c-eb56-419d-beca-6d9cd9f6e62f-maas-api-tls podName:a2fca19c-eb56-419d-beca-6d9cd9f6e62f nodeName:}" failed. No retries permitted until 2026-07-26 18:18:53.441876704 +0000 UTC m=+1790.094067556 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a2fca19c-eb56-419d-beca-6d9cd9f6e62f-maas-api-tls") pod "maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr" (UID: "a2fca19c-eb56-419d-beca-6d9cd9f6e62f") : secret "maas-api-e2e-ait-10cd5747-serving-cert" not found
Jul 26 18:18:52.942265 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:52.942249    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2fca19c-eb56-419d-beca-6d9cd9f6e62f-openshift-service-ca\") pod \"maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr\" (UID: \"a2fca19c-eb56-419d-beca-6d9cd9f6e62f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr"
Jul 26 18:18:52.950914 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:52.950892    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt2cl\" (UniqueName: \"kubernetes.io/projected/a2fca19c-eb56-419d-beca-6d9cd9f6e62f-kube-api-access-rt2cl\") pod \"maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr\" (UID: \"a2fca19c-eb56-419d-beca-6d9cd9f6e62f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr"
Jul 26 18:18:53.445962 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.445925    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2fca19c-eb56-419d-beca-6d9cd9f6e62f-maas-api-tls\") pod \"maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr\" (UID: \"a2fca19c-eb56-419d-beca-6d9cd9f6e62f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr"
Jul 26 18:18:53.448392 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.448364    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2fca19c-eb56-419d-beca-6d9cd9f6e62f-maas-api-tls\") pod \"maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr\" (UID: \"a2fca19c-eb56-419d-beca-6d9cd9f6e62f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr"
Jul 26 18:18:53.482522 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.482464    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z"]
Jul 26 18:18:53.485988 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.485972    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z"
Jul 26 18:18:53.488340 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.488320    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-10cd5747-dockercfg-th7nr\""
Jul 26 18:18:53.488340 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.488331    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-10cd5747\""
Jul 26 18:18:53.497575 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.497550    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z"]
Jul 26 18:18:53.606003 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.605973    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr"
Jul 26 18:18:53.606442 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.606421    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8"]
Jul 26 18:18:53.609693 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.609675    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8"
Jul 26 18:18:53.623649 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.623615    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8"]
Jul 26 18:18:53.648281 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.648249    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/378b05fe-988d-4c5f-bef0-36325cf49b1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z\" (UID: \"378b05fe-988d-4c5f-bef0-36325cf49b1e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z"
Jul 26 18:18:53.648452 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.648365    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw7gk\" (UniqueName: \"kubernetes.io/projected/378b05fe-988d-4c5f-bef0-36325cf49b1e-kube-api-access-lw7gk\") pod \"payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z\" (UID: \"378b05fe-988d-4c5f-bef0-36325cf49b1e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z"
Jul 26 18:18:53.748805 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.748770    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr"]
Jul 26 18:18:53.749274 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.749250    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/781e283b-2b8d-46fd-91bb-38fa9077846e-plugins-config-volume\") pod \"payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8\" (UID: \"781e283b-2b8d-46fd-91bb-38fa9077846e\") " pod="openshift-ingress/payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8"
Jul 26 18:18:53.749392 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.749372    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw7gk\" (UniqueName: \"kubernetes.io/projected/378b05fe-988d-4c5f-bef0-36325cf49b1e-kube-api-access-lw7gk\") pod \"payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z\" (UID: \"378b05fe-988d-4c5f-bef0-36325cf49b1e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z"
Jul 26 18:18:53.749480 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.749412    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clrkl\" (UniqueName: \"kubernetes.io/projected/781e283b-2b8d-46fd-91bb-38fa9077846e-kube-api-access-clrkl\") pod \"payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8\" (UID: \"781e283b-2b8d-46fd-91bb-38fa9077846e\") " pod="openshift-ingress/payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8"
Jul 26 18:18:53.749545 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.749488    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/378b05fe-988d-4c5f-bef0-36325cf49b1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z\" (UID: \"378b05fe-988d-4c5f-bef0-36325cf49b1e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z"
Jul 26 18:18:53.749813 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:18:53.749793    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2fca19c_eb56_419d_beca_6d9cd9f6e62f.slice/crio-fcda460baf7142b9ef5e5ce3105d867838755e1905e6e738d24ee2b2e66b182e WatchSource:0}: Error finding container fcda460baf7142b9ef5e5ce3105d867838755e1905e6e738d24ee2b2e66b182e: Status 404 returned error can't find the container with id fcda460baf7142b9ef5e5ce3105d867838755e1905e6e738d24ee2b2e66b182e
Jul 26 18:18:53.750168 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.750150    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/378b05fe-988d-4c5f-bef0-36325cf49b1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z\" (UID: \"378b05fe-988d-4c5f-bef0-36325cf49b1e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z"
Jul 26 18:18:53.758421 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.758401    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw7gk\" (UniqueName: \"kubernetes.io/projected/378b05fe-988d-4c5f-bef0-36325cf49b1e-kube-api-access-lw7gk\") pod \"payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z\" (UID: \"378b05fe-988d-4c5f-bef0-36325cf49b1e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z"
Jul 26 18:18:53.797699 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.797666    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z"
Jul 26 18:18:53.850616 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.850549    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clrkl\" (UniqueName: \"kubernetes.io/projected/781e283b-2b8d-46fd-91bb-38fa9077846e-kube-api-access-clrkl\") pod \"payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8\" (UID: \"781e283b-2b8d-46fd-91bb-38fa9077846e\") " pod="openshift-ingress/payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8"
Jul 26 18:18:53.850707 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.850652    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/781e283b-2b8d-46fd-91bb-38fa9077846e-plugins-config-volume\") pod \"payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8\" (UID: \"781e283b-2b8d-46fd-91bb-38fa9077846e\") " pod="openshift-ingress/payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8"
Jul 26 18:18:53.851609 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.851585    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/781e283b-2b8d-46fd-91bb-38fa9077846e-plugins-config-volume\") pod \"payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8\" (UID: \"781e283b-2b8d-46fd-91bb-38fa9077846e\") " pod="openshift-ingress/payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8"
Jul 26 18:18:53.860250 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.860186    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clrkl\" (UniqueName: \"kubernetes.io/projected/781e283b-2b8d-46fd-91bb-38fa9077846e-kube-api-access-clrkl\") pod \"payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8\" (UID: \"781e283b-2b8d-46fd-91bb-38fa9077846e\") " pod="openshift-ingress/payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8"
Jul 26 18:18:53.930707 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.930671    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z"]
Jul 26 18:18:53.932065 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:18:53.932028    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod378b05fe_988d_4c5f_bef0_36325cf49b1e.slice/crio-a32fedbb995090e02b975373939ba43bb5042a22b0c554e5f24a40c6453bcc10 WatchSource:0}: Error finding container a32fedbb995090e02b975373939ba43bb5042a22b0c554e5f24a40c6453bcc10: Status 404 returned error can't find the container with id a32fedbb995090e02b975373939ba43bb5042a22b0c554e5f24a40c6453bcc10
Jul 26 18:18:53.936885 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:53.936862    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8"
Jul 26 18:18:54.077394 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:54.077369    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8"]
Jul 26 18:18:54.079095 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:18:54.079052    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod781e283b_2b8d_46fd_91bb_38fa9077846e.slice/crio-5d9b3f75866ba92a29089f61452a3e85d43fb5386812caa21a13d8a26d084a0a WatchSource:0}: Error finding container 5d9b3f75866ba92a29089f61452a3e85d43fb5386812caa21a13d8a26d084a0a: Status 404 returned error can't find the container with id 5d9b3f75866ba92a29089f61452a3e85d43fb5386812caa21a13d8a26d084a0a
Jul 26 18:18:54.098944 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:54.098918    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8" event={"ID":"781e283b-2b8d-46fd-91bb-38fa9077846e","Type":"ContainerStarted","Data":"5d9b3f75866ba92a29089f61452a3e85d43fb5386812caa21a13d8a26d084a0a"}
Jul 26 18:18:54.100724 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:54.100692    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z" event={"ID":"378b05fe-988d-4c5f-bef0-36325cf49b1e","Type":"ContainerStarted","Data":"e8ca7dfed1736a40c452d9cce79c363eaf5586d693023c23c24914927578a8f9"}
Jul 26 18:18:54.100837 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:54.100755    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z" event={"ID":"378b05fe-988d-4c5f-bef0-36325cf49b1e","Type":"ContainerStarted","Data":"a32fedbb995090e02b975373939ba43bb5042a22b0c554e5f24a40c6453bcc10"}
Jul 26 18:18:54.100837 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:54.100791    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z"
Jul 26 18:18:54.101915 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:54.101892    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr" event={"ID":"a2fca19c-eb56-419d-beca-6d9cd9f6e62f","Type":"ContainerStarted","Data":"fcda460baf7142b9ef5e5ce3105d867838755e1905e6e738d24ee2b2e66b182e"}
Jul 26 18:18:54.118515 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:54.118462    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z" podStartSLOduration=1.118449663 podStartE2EDuration="1.118449663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:18:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:18:54.11628496 +0000 UTC m=+1790.768475822" watchObservedRunningTime="2026-07-26 18:18:54.118449663 +0000 UTC m=+1790.770640520"
Jul 26 18:18:55.108831 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:55.108793    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8" event={"ID":"781e283b-2b8d-46fd-91bb-38fa9077846e","Type":"ContainerStarted","Data":"a3c9422e17800f2f8f81626021b0b3feba3ca00d5b8e5187af981e2cc06fba56"}
Jul 26 18:18:55.109293 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:55.108942    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8"
Jul 26 18:18:55.110489 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:55.110467    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr" event={"ID":"a2fca19c-eb56-419d-beca-6d9cd9f6e62f","Type":"ContainerStarted","Data":"b1bd890a4595d40613e5bc728b47b316097b7cab76d6cf009fca2320d0ff48d9"}
Jul 26 18:18:55.110626 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:55.110612    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr"
Jul 26 18:18:55.126839 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:55.126789    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8" podStartSLOduration=2.126773136 podStartE2EDuration="2.126773136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:18:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:18:55.126353006 +0000 UTC m=+1791.778543865" watchObservedRunningTime="2026-07-26 18:18:55.126773136 +0000 UTC m=+1791.778964022"
Jul 26 18:18:55.146192 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:18:55.146141    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr" podStartSLOduration=2.6966311689999998 podStartE2EDuration="3.146126292s" totalImagesPullingTime="449.495123ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:18:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:18:55.143440579 +0000 UTC m=+1791.795631439" watchObservedRunningTime="2026-07-26 18:18:55.146126292 +0000 UTC m=+1791.798317140"
Jul 26 18:19:01.121888 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:01.121805    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr"
Jul 26 18:19:03.946000 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:03.945971    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 18:19:03.949680 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:03.949658    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 18:19:05.112322 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:05.112290    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z"
Jul 26 18:19:06.122551 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:06.122514    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8"
Jul 26 18:19:18.275518 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:18.275481    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10cd5747-4gjpf"]
Jul 26 18:19:18.278935 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:18.278918    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10cd5747-4gjpf"
Jul 26 18:19:18.281092 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:18.281075    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-swzk5\""
Jul 26 18:19:18.285703 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:18.285672    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10cd5747-4gjpf"]
Jul 26 18:19:18.363915 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:18.363877    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1338cfd-82d1-470e-8c2a-de3d2a37dc02-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10cd5747-4gjpf\" (UID: \"c1338cfd-82d1-470e-8c2a-de3d2a37dc02\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10cd5747-4gjpf"
Jul 26 18:19:18.464518 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:18.464478    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1338cfd-82d1-470e-8c2a-de3d2a37dc02-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10cd5747-4gjpf\" (UID: \"c1338cfd-82d1-470e-8c2a-de3d2a37dc02\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10cd5747-4gjpf"
Jul 26 18:19:18.465171 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:18.465153    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1338cfd-82d1-470e-8c2a-de3d2a37dc02-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10cd5747-4gjpf\" (UID: \"c1338cfd-82d1-470e-8c2a-de3d2a37dc02\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10cd5747-4gjpf"
Jul 26 18:19:18.591776 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:18.591672    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10cd5747-4gjpf"
Jul 26 18:19:18.723900 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:18.723867    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10cd5747-4gjpf"]
Jul 26 18:19:18.725360 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:19:18.725329    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1338cfd_82d1_470e_8c2a_de3d2a37dc02.slice/crio-2e6116fbfad46defd68e0d383620f8aa4dc98af99b211788d4739c8bbd82f033 WatchSource:0}: Error finding container 2e6116fbfad46defd68e0d383620f8aa4dc98af99b211788d4739c8bbd82f033: Status 404 returned error can't find the container with id 2e6116fbfad46defd68e0d383620f8aa4dc98af99b211788d4739c8bbd82f033
Jul 26 18:19:19.245738 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:19.245649    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10cd5747-4gjpf" event={"ID":"c1338cfd-82d1-470e-8c2a-de3d2a37dc02","Type":"ContainerStarted","Data":"4ced9639cd1e5ee5c3ff6bebab08726621c71c0260d9a649795a0c37bfd5553f"}
Jul 26 18:19:19.245738 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:19.245706    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10cd5747-4gjpf" event={"ID":"c1338cfd-82d1-470e-8c2a-de3d2a37dc02","Type":"ContainerStarted","Data":"2e6116fbfad46defd68e0d383620f8aa4dc98af99b211788d4739c8bbd82f033"}
Jul 26 18:19:19.259627 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:19.259577    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10cd5747-4gjpf" podStartSLOduration=1.259564045 podStartE2EDuration="1.259564045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:19:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:19:19.258314016 +0000 UTC m=+1815.910504875" watchObservedRunningTime="2026-07-26 18:19:19.259564045 +0000 UTC m=+1815.911754902"
Jul 26 18:19:20.252401 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:20.252363    2570 generic.go:356] "Generic (PLEG): container finished" podID="c1338cfd-82d1-470e-8c2a-de3d2a37dc02" containerID="4ced9639cd1e5ee5c3ff6bebab08726621c71c0260d9a649795a0c37bfd5553f" exitCode=0
Jul 26 18:19:20.252882 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:20.252450    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10cd5747-4gjpf" event={"ID":"c1338cfd-82d1-470e-8c2a-de3d2a37dc02","Type":"ContainerDied","Data":"4ced9639cd1e5ee5c3ff6bebab08726621c71c0260d9a649795a0c37bfd5553f"}
Jul 26 18:19:21.396711 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:21.396683    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10cd5747-4gjpf"
Jul 26 18:19:21.490506 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:21.490468    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1338cfd-82d1-470e-8c2a-de3d2a37dc02-maas-api-service-ca\") pod \"c1338cfd-82d1-470e-8c2a-de3d2a37dc02\" (UID: \"c1338cfd-82d1-470e-8c2a-de3d2a37dc02\") "
Jul 26 18:19:21.490883 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:21.490847    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1338cfd-82d1-470e-8c2a-de3d2a37dc02-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c1338cfd-82d1-470e-8c2a-de3d2a37dc02" (UID: "c1338cfd-82d1-470e-8c2a-de3d2a37dc02"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:19:21.591388 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:21.591300    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1338cfd-82d1-470e-8c2a-de3d2a37dc02-maas-api-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:19:22.265524 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:22.265494    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10cd5747-4gjpf"
Jul 26 18:19:22.265704 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:22.265521    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10cd5747-4gjpf" event={"ID":"c1338cfd-82d1-470e-8c2a-de3d2a37dc02","Type":"ContainerDied","Data":"2e6116fbfad46defd68e0d383620f8aa4dc98af99b211788d4739c8bbd82f033"}
Jul 26 18:19:22.265704 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:22.265567    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e6116fbfad46defd68e0d383620f8aa4dc98af99b211788d4739c8bbd82f033"
Jul 26 18:19:28.671040 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:28.670997    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr"]
Jul 26 18:19:28.671473 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:28.671281    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr" podUID="a2fca19c-eb56-419d-beca-6d9cd9f6e62f" containerName="maas-api" containerID="cri-o://b1bd890a4595d40613e5bc728b47b316097b7cab76d6cf009fca2320d0ff48d9" gracePeriod=30
Jul 26 18:19:28.730287 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:28.730255    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8"]
Jul 26 18:19:28.730505 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:28.730480    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8" podUID="781e283b-2b8d-46fd-91bb-38fa9077846e" containerName="payload-processing" containerID="cri-o://a3c9422e17800f2f8f81626021b0b3feba3ca00d5b8e5187af981e2cc06fba56" gracePeriod=30
Jul 26 18:19:28.736828 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:28.736790    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z"]
Jul 26 18:19:28.737065 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:28.737016    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z" podUID="378b05fe-988d-4c5f-bef0-36325cf49b1e" containerName="payload-pre-processing" containerID="cri-o://e8ca7dfed1736a40c452d9cce79c363eaf5586d693023c23c24914927578a8f9" gracePeriod=30
Jul 26 18:19:29.000504 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.000478    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr"
Jul 26 18:19:29.043409 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.043383    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8"
Jul 26 18:19:29.046252 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.046232    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z"
Jul 26 18:19:29.056483 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.056461    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2fca19c-eb56-419d-beca-6d9cd9f6e62f-openshift-service-ca\") pod \"a2fca19c-eb56-419d-beca-6d9cd9f6e62f\" (UID: \"a2fca19c-eb56-419d-beca-6d9cd9f6e62f\") "
Jul 26 18:19:29.056579 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.056566    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2fca19c-eb56-419d-beca-6d9cd9f6e62f-maas-api-tls\") pod \"a2fca19c-eb56-419d-beca-6d9cd9f6e62f\" (UID: \"a2fca19c-eb56-419d-beca-6d9cd9f6e62f\") "
Jul 26 18:19:29.056652 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.056604    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rt2cl\" (UniqueName: \"kubernetes.io/projected/a2fca19c-eb56-419d-beca-6d9cd9f6e62f-kube-api-access-rt2cl\") pod \"a2fca19c-eb56-419d-beca-6d9cd9f6e62f\" (UID: \"a2fca19c-eb56-419d-beca-6d9cd9f6e62f\") "
Jul 26 18:19:29.056852 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.056815    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2fca19c-eb56-419d-beca-6d9cd9f6e62f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a2fca19c-eb56-419d-beca-6d9cd9f6e62f" (UID: "a2fca19c-eb56-419d-beca-6d9cd9f6e62f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:19:29.058638 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.058612    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2fca19c-eb56-419d-beca-6d9cd9f6e62f-kube-api-access-rt2cl" (OuterVolumeSpecName: "kube-api-access-rt2cl") pod "a2fca19c-eb56-419d-beca-6d9cd9f6e62f" (UID: "a2fca19c-eb56-419d-beca-6d9cd9f6e62f"). InnerVolumeSpecName "kube-api-access-rt2cl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:19:29.058638 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.058626    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2fca19c-eb56-419d-beca-6d9cd9f6e62f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a2fca19c-eb56-419d-beca-6d9cd9f6e62f" (UID: "a2fca19c-eb56-419d-beca-6d9cd9f6e62f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:19:29.157974 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.157936    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clrkl\" (UniqueName: \"kubernetes.io/projected/781e283b-2b8d-46fd-91bb-38fa9077846e-kube-api-access-clrkl\") pod \"781e283b-2b8d-46fd-91bb-38fa9077846e\" (UID: \"781e283b-2b8d-46fd-91bb-38fa9077846e\") "
Jul 26 18:19:29.157974 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.157982    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lw7gk\" (UniqueName: \"kubernetes.io/projected/378b05fe-988d-4c5f-bef0-36325cf49b1e-kube-api-access-lw7gk\") pod \"378b05fe-988d-4c5f-bef0-36325cf49b1e\" (UID: \"378b05fe-988d-4c5f-bef0-36325cf49b1e\") "
Jul 26 18:19:29.158255 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.158022    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/781e283b-2b8d-46fd-91bb-38fa9077846e-plugins-config-volume\") pod \"781e283b-2b8d-46fd-91bb-38fa9077846e\" (UID: \"781e283b-2b8d-46fd-91bb-38fa9077846e\") "
Jul 26 18:19:29.158255 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.158066    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/378b05fe-988d-4c5f-bef0-36325cf49b1e-plugins-config-volume\") pod \"378b05fe-988d-4c5f-bef0-36325cf49b1e\" (UID: \"378b05fe-988d-4c5f-bef0-36325cf49b1e\") "
Jul 26 18:19:29.158443 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.158418    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2fca19c-eb56-419d-beca-6d9cd9f6e62f-maas-api-tls\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:19:29.158504 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.158451    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rt2cl\" (UniqueName: \"kubernetes.io/projected/a2fca19c-eb56-419d-beca-6d9cd9f6e62f-kube-api-access-rt2cl\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:19:29.158504 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.158469    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2fca19c-eb56-419d-beca-6d9cd9f6e62f-openshift-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:19:29.158504 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.158471    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/378b05fe-988d-4c5f-bef0-36325cf49b1e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "378b05fe-988d-4c5f-bef0-36325cf49b1e" (UID: "378b05fe-988d-4c5f-bef0-36325cf49b1e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:19:29.158504 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.158471    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/781e283b-2b8d-46fd-91bb-38fa9077846e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "781e283b-2b8d-46fd-91bb-38fa9077846e" (UID: "781e283b-2b8d-46fd-91bb-38fa9077846e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:19:29.160097 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.160076    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/378b05fe-988d-4c5f-bef0-36325cf49b1e-kube-api-access-lw7gk" (OuterVolumeSpecName: "kube-api-access-lw7gk") pod "378b05fe-988d-4c5f-bef0-36325cf49b1e" (UID: "378b05fe-988d-4c5f-bef0-36325cf49b1e"). InnerVolumeSpecName "kube-api-access-lw7gk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:19:29.160344 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.160328    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/781e283b-2b8d-46fd-91bb-38fa9077846e-kube-api-access-clrkl" (OuterVolumeSpecName: "kube-api-access-clrkl") pod "781e283b-2b8d-46fd-91bb-38fa9077846e" (UID: "781e283b-2b8d-46fd-91bb-38fa9077846e"). InnerVolumeSpecName "kube-api-access-clrkl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:19:29.259492 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.259466    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clrkl\" (UniqueName: \"kubernetes.io/projected/781e283b-2b8d-46fd-91bb-38fa9077846e-kube-api-access-clrkl\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:19:29.259492 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.259492    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lw7gk\" (UniqueName: \"kubernetes.io/projected/378b05fe-988d-4c5f-bef0-36325cf49b1e-kube-api-access-lw7gk\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:19:29.259682 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.259501    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/781e283b-2b8d-46fd-91bb-38fa9077846e-plugins-config-volume\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:19:29.259682 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.259512    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/378b05fe-988d-4c5f-bef0-36325cf49b1e-plugins-config-volume\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:19:29.307534 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.307497    2570 generic.go:356] "Generic (PLEG): container finished" podID="781e283b-2b8d-46fd-91bb-38fa9077846e" containerID="a3c9422e17800f2f8f81626021b0b3feba3ca00d5b8e5187af981e2cc06fba56" exitCode=0
Jul 26 18:19:29.307731 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.307543    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8"
Jul 26 18:19:29.307731 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.307577    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8" event={"ID":"781e283b-2b8d-46fd-91bb-38fa9077846e","Type":"ContainerDied","Data":"a3c9422e17800f2f8f81626021b0b3feba3ca00d5b8e5187af981e2cc06fba56"}
Jul 26 18:19:29.307731 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.307655    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8" event={"ID":"781e283b-2b8d-46fd-91bb-38fa9077846e","Type":"ContainerDied","Data":"5d9b3f75866ba92a29089f61452a3e85d43fb5386812caa21a13d8a26d084a0a"}
Jul 26 18:19:29.307731 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.307687    2570 scope.go:117] "RemoveContainer" containerID="a3c9422e17800f2f8f81626021b0b3feba3ca00d5b8e5187af981e2cc06fba56"
Jul 26 18:19:29.308896 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.308870    2570 generic.go:356] "Generic (PLEG): container finished" podID="378b05fe-988d-4c5f-bef0-36325cf49b1e" containerID="e8ca7dfed1736a40c452d9cce79c363eaf5586d693023c23c24914927578a8f9" exitCode=0
Jul 26 18:19:29.308994 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.308912    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z"
Jul 26 18:19:29.308994 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.308963    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z" event={"ID":"378b05fe-988d-4c5f-bef0-36325cf49b1e","Type":"ContainerDied","Data":"e8ca7dfed1736a40c452d9cce79c363eaf5586d693023c23c24914927578a8f9"}
Jul 26 18:19:29.309086 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.309016    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z" event={"ID":"378b05fe-988d-4c5f-bef0-36325cf49b1e","Type":"ContainerDied","Data":"a32fedbb995090e02b975373939ba43bb5042a22b0c554e5f24a40c6453bcc10"}
Jul 26 18:19:29.310631 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.310602    2570 generic.go:356] "Generic (PLEG): container finished" podID="a2fca19c-eb56-419d-beca-6d9cd9f6e62f" containerID="b1bd890a4595d40613e5bc728b47b316097b7cab76d6cf009fca2320d0ff48d9" exitCode=0
Jul 26 18:19:29.310762 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.310688    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr" event={"ID":"a2fca19c-eb56-419d-beca-6d9cd9f6e62f","Type":"ContainerDied","Data":"b1bd890a4595d40613e5bc728b47b316097b7cab76d6cf009fca2320d0ff48d9"}
Jul 26 18:19:29.310762 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.310698    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr"
Jul 26 18:19:29.310762 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.310726    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr" event={"ID":"a2fca19c-eb56-419d-beca-6d9cd9f6e62f","Type":"ContainerDied","Data":"fcda460baf7142b9ef5e5ce3105d867838755e1905e6e738d24ee2b2e66b182e"}
Jul 26 18:19:29.318230 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.318212    2570 scope.go:117] "RemoveContainer" containerID="a3c9422e17800f2f8f81626021b0b3feba3ca00d5b8e5187af981e2cc06fba56"
Jul 26 18:19:29.318525 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:19:29.318504    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3c9422e17800f2f8f81626021b0b3feba3ca00d5b8e5187af981e2cc06fba56\": container with ID starting with a3c9422e17800f2f8f81626021b0b3feba3ca00d5b8e5187af981e2cc06fba56 not found: ID does not exist" containerID="a3c9422e17800f2f8f81626021b0b3feba3ca00d5b8e5187af981e2cc06fba56"
Jul 26 18:19:29.318607 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.318533    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3c9422e17800f2f8f81626021b0b3feba3ca00d5b8e5187af981e2cc06fba56"} err="failed to get container status \"a3c9422e17800f2f8f81626021b0b3feba3ca00d5b8e5187af981e2cc06fba56\": rpc error: code = NotFound desc = could not find container \"a3c9422e17800f2f8f81626021b0b3feba3ca00d5b8e5187af981e2cc06fba56\": container with ID starting with a3c9422e17800f2f8f81626021b0b3feba3ca00d5b8e5187af981e2cc06fba56 not found: ID does not exist"
Jul 26 18:19:29.318607 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.318551    2570 scope.go:117] "RemoveContainer" containerID="e8ca7dfed1736a40c452d9cce79c363eaf5586d693023c23c24914927578a8f9"
Jul 26 18:19:29.327377 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.327360    2570 scope.go:117] "RemoveContainer" containerID="e8ca7dfed1736a40c452d9cce79c363eaf5586d693023c23c24914927578a8f9"
Jul 26 18:19:29.327624 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:19:29.327605    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8ca7dfed1736a40c452d9cce79c363eaf5586d693023c23c24914927578a8f9\": container with ID starting with e8ca7dfed1736a40c452d9cce79c363eaf5586d693023c23c24914927578a8f9 not found: ID does not exist" containerID="e8ca7dfed1736a40c452d9cce79c363eaf5586d693023c23c24914927578a8f9"
Jul 26 18:19:29.327673 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.327631    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8ca7dfed1736a40c452d9cce79c363eaf5586d693023c23c24914927578a8f9"} err="failed to get container status \"e8ca7dfed1736a40c452d9cce79c363eaf5586d693023c23c24914927578a8f9\": rpc error: code = NotFound desc = could not find container \"e8ca7dfed1736a40c452d9cce79c363eaf5586d693023c23c24914927578a8f9\": container with ID starting with e8ca7dfed1736a40c452d9cce79c363eaf5586d693023c23c24914927578a8f9 not found: ID does not exist"
Jul 26 18:19:29.327673 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.327649    2570 scope.go:117] "RemoveContainer" containerID="b1bd890a4595d40613e5bc728b47b316097b7cab76d6cf009fca2320d0ff48d9"
Jul 26 18:19:29.331972 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.331952    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8"]
Jul 26 18:19:29.335272 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.335226    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-10cd5747-7d985cf7c4-sbwc8"]
Jul 26 18:19:29.335818 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.335802    2570 scope.go:117] "RemoveContainer" containerID="b1bd890a4595d40613e5bc728b47b316097b7cab76d6cf009fca2320d0ff48d9"
Jul 26 18:19:29.336099 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:19:29.336080    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1bd890a4595d40613e5bc728b47b316097b7cab76d6cf009fca2320d0ff48d9\": container with ID starting with b1bd890a4595d40613e5bc728b47b316097b7cab76d6cf009fca2320d0ff48d9 not found: ID does not exist" containerID="b1bd890a4595d40613e5bc728b47b316097b7cab76d6cf009fca2320d0ff48d9"
Jul 26 18:19:29.336212 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.336104    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1bd890a4595d40613e5bc728b47b316097b7cab76d6cf009fca2320d0ff48d9"} err="failed to get container status \"b1bd890a4595d40613e5bc728b47b316097b7cab76d6cf009fca2320d0ff48d9\": rpc error: code = NotFound desc = could not find container \"b1bd890a4595d40613e5bc728b47b316097b7cab76d6cf009fca2320d0ff48d9\": container with ID starting with b1bd890a4595d40613e5bc728b47b316097b7cab76d6cf009fca2320d0ff48d9 not found: ID does not exist"
Jul 26 18:19:29.345349 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.345327    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr"]
Jul 26 18:19:29.347409 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.347385    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-10cd5747-78f59cdf8d-9gkmr"]
Jul 26 18:19:29.356613 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.356593    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z"]
Jul 26 18:19:29.358581 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.358563    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-10cd5747-79678f69cb-kxs6z"]
Jul 26 18:19:29.838267 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.838238    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="378b05fe-988d-4c5f-bef0-36325cf49b1e" path="/var/lib/kubelet/pods/378b05fe-988d-4c5f-bef0-36325cf49b1e/volumes"
Jul 26 18:19:29.838648 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.838600    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="781e283b-2b8d-46fd-91bb-38fa9077846e" path="/var/lib/kubelet/pods/781e283b-2b8d-46fd-91bb-38fa9077846e/volumes"
Jul 26 18:19:29.838929 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:29.838916    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2fca19c-eb56-419d-beca-6d9cd9f6e62f" path="/var/lib/kubelet/pods/a2fca19c-eb56-419d-beca-6d9cd9f6e62f/volumes"
Jul 26 18:19:33.384824 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:33.384784    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10cd5747-4gjpf"]
Jul 26 18:19:33.389164 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:33.389137    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10cd5747-4gjpf"]
Jul 26 18:19:33.838928 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:33.838882    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1338cfd-82d1-470e-8c2a-de3d2a37dc02" path="/var/lib/kubelet/pods/c1338cfd-82d1-470e-8c2a-de3d2a37dc02/volumes"
Jul 26 18:19:37.939492 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:37.939458    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv"]
Jul 26 18:19:37.939959 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:37.939804    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="378b05fe-988d-4c5f-bef0-36325cf49b1e" containerName="payload-pre-processing"
Jul 26 18:19:37.939959 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:37.939815    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="378b05fe-988d-4c5f-bef0-36325cf49b1e" containerName="payload-pre-processing"
Jul 26 18:19:37.939959 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:37.939822    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="781e283b-2b8d-46fd-91bb-38fa9077846e" containerName="payload-processing"
Jul 26 18:19:37.939959 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:37.939828    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="781e283b-2b8d-46fd-91bb-38fa9077846e" containerName="payload-processing"
Jul 26 18:19:37.939959 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:37.939846    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2fca19c-eb56-419d-beca-6d9cd9f6e62f" containerName="maas-api"
Jul 26 18:19:37.939959 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:37.939852    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2fca19c-eb56-419d-beca-6d9cd9f6e62f" containerName="maas-api"
Jul 26 18:19:37.939959 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:37.939860    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1338cfd-82d1-470e-8c2a-de3d2a37dc02" containerName="revoke-keys"
Jul 26 18:19:37.939959 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:37.939865    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1338cfd-82d1-470e-8c2a-de3d2a37dc02" containerName="revoke-keys"
Jul 26 18:19:37.939959 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:37.939922    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="781e283b-2b8d-46fd-91bb-38fa9077846e" containerName="payload-processing"
Jul 26 18:19:37.939959 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:37.939929    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="378b05fe-988d-4c5f-bef0-36325cf49b1e" containerName="payload-pre-processing"
Jul 26 18:19:37.939959 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:37.939935    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1338cfd-82d1-470e-8c2a-de3d2a37dc02" containerName="revoke-keys"
Jul 26 18:19:37.939959 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:37.939942    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2fca19c-eb56-419d-beca-6d9cd9f6e62f" containerName="maas-api"
Jul 26 18:19:37.942890 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:37.942874    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv"
Jul 26 18:19:37.964153 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:37.964130    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-535bf7ee-serving-cert\""
Jul 26 18:19:37.981305 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:37.981278    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv"]
Jul 26 18:19:38.037639 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.037606    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8gk8\" (UniqueName: \"kubernetes.io/projected/7f64bf52-840f-49b5-9016-55d214983935-kube-api-access-q8gk8\") pod \"maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv\" (UID: \"7f64bf52-840f-49b5-9016-55d214983935\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv"
Jul 26 18:19:38.037812 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.037656    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f64bf52-840f-49b5-9016-55d214983935-maas-api-tls\") pod \"maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv\" (UID: \"7f64bf52-840f-49b5-9016-55d214983935\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv"
Jul 26 18:19:38.037812 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.037730    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f64bf52-840f-49b5-9016-55d214983935-openshift-service-ca\") pod \"maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv\" (UID: \"7f64bf52-840f-49b5-9016-55d214983935\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv"
Jul 26 18:19:38.138734 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.138701    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f64bf52-840f-49b5-9016-55d214983935-openshift-service-ca\") pod \"maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv\" (UID: \"7f64bf52-840f-49b5-9016-55d214983935\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv"
Jul 26 18:19:38.138908 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.138767    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8gk8\" (UniqueName: \"kubernetes.io/projected/7f64bf52-840f-49b5-9016-55d214983935-kube-api-access-q8gk8\") pod \"maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv\" (UID: \"7f64bf52-840f-49b5-9016-55d214983935\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv"
Jul 26 18:19:38.138908 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.138802    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f64bf52-840f-49b5-9016-55d214983935-maas-api-tls\") pod \"maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv\" (UID: \"7f64bf52-840f-49b5-9016-55d214983935\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv"
Jul 26 18:19:38.138908 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:19:38.138903    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-serving-cert: secret "maas-api-e2e-derive-535bf7ee-serving-cert" not found
Jul 26 18:19:38.139015 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:19:38.138959    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f64bf52-840f-49b5-9016-55d214983935-maas-api-tls podName:7f64bf52-840f-49b5-9016-55d214983935 nodeName:}" failed. No retries permitted until 2026-07-26 18:19:38.638943889 +0000 UTC m=+1835.291134725 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7f64bf52-840f-49b5-9016-55d214983935-maas-api-tls") pod "maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv" (UID: "7f64bf52-840f-49b5-9016-55d214983935") : secret "maas-api-e2e-derive-535bf7ee-serving-cert" not found
Jul 26 18:19:38.139385 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.139363    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f64bf52-840f-49b5-9016-55d214983935-openshift-service-ca\") pod \"maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv\" (UID: \"7f64bf52-840f-49b5-9016-55d214983935\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv"
Jul 26 18:19:38.147040 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.147018    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8gk8\" (UniqueName: \"kubernetes.io/projected/7f64bf52-840f-49b5-9016-55d214983935-kube-api-access-q8gk8\") pod \"maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv\" (UID: \"7f64bf52-840f-49b5-9016-55d214983935\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv"
Jul 26 18:19:38.644468 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.644429    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f64bf52-840f-49b5-9016-55d214983935-maas-api-tls\") pod \"maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv\" (UID: \"7f64bf52-840f-49b5-9016-55d214983935\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv"
Jul 26 18:19:38.646852 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.646824    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f64bf52-840f-49b5-9016-55d214983935-maas-api-tls\") pod \"maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv\" (UID: \"7f64bf52-840f-49b5-9016-55d214983935\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv"
Jul 26 18:19:38.706668 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.706632    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272"]
Jul 26 18:19:38.710235 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.710215    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272"
Jul 26 18:19:38.713049 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.713028    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-535bf7ee\""
Jul 26 18:19:38.728567 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.728541    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272"]
Jul 26 18:19:38.738527 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.738501    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk"]
Jul 26 18:19:38.742479 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.742460    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk"
Jul 26 18:19:38.744637 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.744618    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-535bf7ee-dockercfg-7bhgb\""
Jul 26 18:19:38.752651 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.752628    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk"]
Jul 26 18:19:38.845287 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.845247    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trvmc\" (UniqueName: \"kubernetes.io/projected/55768422-a4b0-4260-a630-1918eab0030d-kube-api-access-trvmc\") pod \"payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk\" (UID: \"55768422-a4b0-4260-a630-1918eab0030d\") " pod="openshift-ingress/payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk"
Jul 26 18:19:38.845465 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.845309    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55768422-a4b0-4260-a630-1918eab0030d-plugins-config-volume\") pod \"payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk\" (UID: \"55768422-a4b0-4260-a630-1918eab0030d\") " pod="openshift-ingress/payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk"
Jul 26 18:19:38.845465 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.845369    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a3318fa-29b2-4ea8-8a32-ff9f28457492-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272\" (UID: \"9a3318fa-29b2-4ea8-8a32-ff9f28457492\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272"
Jul 26 18:19:38.845465 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.845407    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gcl6\" (UniqueName: \"kubernetes.io/projected/9a3318fa-29b2-4ea8-8a32-ff9f28457492-kube-api-access-7gcl6\") pod \"payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272\" (UID: \"9a3318fa-29b2-4ea8-8a32-ff9f28457492\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272"
Jul 26 18:19:38.852200 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.852170    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv"
Jul 26 18:19:38.970555 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.960568    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gcl6\" (UniqueName: \"kubernetes.io/projected/9a3318fa-29b2-4ea8-8a32-ff9f28457492-kube-api-access-7gcl6\") pod \"payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272\" (UID: \"9a3318fa-29b2-4ea8-8a32-ff9f28457492\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272"
Jul 26 18:19:38.970555 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.960632    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trvmc\" (UniqueName: \"kubernetes.io/projected/55768422-a4b0-4260-a630-1918eab0030d-kube-api-access-trvmc\") pod \"payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk\" (UID: \"55768422-a4b0-4260-a630-1918eab0030d\") " pod="openshift-ingress/payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk"
Jul 26 18:19:38.970555 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.960711    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55768422-a4b0-4260-a630-1918eab0030d-plugins-config-volume\") pod \"payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk\" (UID: \"55768422-a4b0-4260-a630-1918eab0030d\") " pod="openshift-ingress/payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk"
Jul 26 18:19:38.970555 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.960782    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a3318fa-29b2-4ea8-8a32-ff9f28457492-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272\" (UID: \"9a3318fa-29b2-4ea8-8a32-ff9f28457492\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272"
Jul 26 18:19:38.970555 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.961736    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a3318fa-29b2-4ea8-8a32-ff9f28457492-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272\" (UID: \"9a3318fa-29b2-4ea8-8a32-ff9f28457492\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272"
Jul 26 18:19:38.970555 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.963184    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55768422-a4b0-4260-a630-1918eab0030d-plugins-config-volume\") pod \"payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk\" (UID: \"55768422-a4b0-4260-a630-1918eab0030d\") " pod="openshift-ingress/payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk"
Jul 26 18:19:38.971426 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.971350    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trvmc\" (UniqueName: \"kubernetes.io/projected/55768422-a4b0-4260-a630-1918eab0030d-kube-api-access-trvmc\") pod \"payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk\" (UID: \"55768422-a4b0-4260-a630-1918eab0030d\") " pod="openshift-ingress/payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk"
Jul 26 18:19:38.975322 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:38.975293    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gcl6\" (UniqueName: \"kubernetes.io/projected/9a3318fa-29b2-4ea8-8a32-ff9f28457492-kube-api-access-7gcl6\") pod \"payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272\" (UID: \"9a3318fa-29b2-4ea8-8a32-ff9f28457492\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272"
Jul 26 18:19:39.019700 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:39.019675    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv"]
Jul 26 18:19:39.021239 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:19:39.021211    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f64bf52_840f_49b5_9016_55d214983935.slice/crio-882e24ebe24ce0c9b6b1a2b992739538eb53af59356829a757e7abc85c1e974c WatchSource:0}: Error finding container 882e24ebe24ce0c9b6b1a2b992739538eb53af59356829a757e7abc85c1e974c: Status 404 returned error can't find the container with id 882e24ebe24ce0c9b6b1a2b992739538eb53af59356829a757e7abc85c1e974c
Jul 26 18:19:39.028076 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:39.028041    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272"
Jul 26 18:19:39.053176 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:39.053151    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk"
Jul 26 18:19:39.368927 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:39.368893    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv" event={"ID":"7f64bf52-840f-49b5-9016-55d214983935","Type":"ContainerStarted","Data":"882e24ebe24ce0c9b6b1a2b992739538eb53af59356829a757e7abc85c1e974c"}
Jul 26 18:19:39.375238 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:39.375212    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272"]
Jul 26 18:19:39.408839 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:39.408817    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk"]
Jul 26 18:19:39.411038 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:19:39.411012    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55768422_a4b0_4260_a630_1918eab0030d.slice/crio-f6fe59993411893c59d43f6be77dcf033e532784f044abd40bf989b955973225 WatchSource:0}: Error finding container f6fe59993411893c59d43f6be77dcf033e532784f044abd40bf989b955973225: Status 404 returned error can't find the container with id f6fe59993411893c59d43f6be77dcf033e532784f044abd40bf989b955973225
Jul 26 18:19:40.377354 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:40.377310    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv" event={"ID":"7f64bf52-840f-49b5-9016-55d214983935","Type":"ContainerStarted","Data":"c64f006b893072ec86f02ff3f743fd675e9ddd280eca13e9a90354fe877ee515"}
Jul 26 18:19:40.377856 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:40.377405    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv"
Jul 26 18:19:40.378897 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:40.378875    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk" event={"ID":"55768422-a4b0-4260-a630-1918eab0030d","Type":"ContainerStarted","Data":"b91fa4d33a3a7253f642733af463728a8cddcf7549cb62392de12508995e911b"}
Jul 26 18:19:40.378971 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:40.378916    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk" event={"ID":"55768422-a4b0-4260-a630-1918eab0030d","Type":"ContainerStarted","Data":"f6fe59993411893c59d43f6be77dcf033e532784f044abd40bf989b955973225"}
Jul 26 18:19:40.378971 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:40.378958    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk"
Jul 26 18:19:40.380470 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:40.380445    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272" event={"ID":"9a3318fa-29b2-4ea8-8a32-ff9f28457492","Type":"ContainerStarted","Data":"e32e7dc26b85870db77cb5c9ec545dd16b56ab19ea4328f592e200f3d010e4c0"}
Jul 26 18:19:40.380613 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:40.380489    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272"
Jul 26 18:19:40.380613 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:40.380506    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272" event={"ID":"9a3318fa-29b2-4ea8-8a32-ff9f28457492","Type":"ContainerStarted","Data":"3ffdff2141a414bc2ee9b34b9b1028d77efd2fc4276b9dd200d37c2b580c7901"}
Jul 26 18:19:40.394221 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:40.394169    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv" podStartSLOduration=3.020814353 podStartE2EDuration="3.39415517s" totalImagesPullingTime="373.340817ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:19:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:19:40.392660244 +0000 UTC m=+1837.044851116" watchObservedRunningTime="2026-07-26 18:19:40.39415517 +0000 UTC m=+1837.046346028"
Jul 26 18:19:40.409184 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:40.409138    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272" podStartSLOduration=2.409124661 podStartE2EDuration="2.409124661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:19:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:19:40.406700232 +0000 UTC m=+1837.058891090" watchObservedRunningTime="2026-07-26 18:19:40.409124661 +0000 UTC m=+1837.061315514"
Jul 26 18:19:40.422335 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:40.422295    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk" podStartSLOduration=2.422282988 podStartE2EDuration="2.422282988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:19:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:19:40.420832076 +0000 UTC m=+1837.073022935" watchObservedRunningTime="2026-07-26 18:19:40.422282988 +0000 UTC m=+1837.074473845"
Jul 26 18:19:46.392312 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:46.392277    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv"
Jul 26 18:19:51.388231 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:51.388179    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272"
Jul 26 18:19:51.388701 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:19:51.388382    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk"
Jul 26 18:20:02.633736 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:02.633703    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-535bf7ee-g4g7k"]
Jul 26 18:20:02.638436 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:02.638419    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-535bf7ee-g4g7k"
Jul 26 18:20:02.641005 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:02.640985    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-swzk5\""
Jul 26 18:20:02.646435 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:02.646335    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-535bf7ee-g4g7k"]
Jul 26 18:20:02.661376 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:02.661348    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c109f946-9e81-495e-a102-0061279e3da6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-535bf7ee-g4g7k\" (UID: \"c109f946-9e81-495e-a102-0061279e3da6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-535bf7ee-g4g7k"
Jul 26 18:20:02.762053 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:02.762011    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c109f946-9e81-495e-a102-0061279e3da6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-535bf7ee-g4g7k\" (UID: \"c109f946-9e81-495e-a102-0061279e3da6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-535bf7ee-g4g7k"
Jul 26 18:20:02.762660 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:02.762638    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c109f946-9e81-495e-a102-0061279e3da6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-535bf7ee-g4g7k\" (UID: \"c109f946-9e81-495e-a102-0061279e3da6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-535bf7ee-g4g7k"
Jul 26 18:20:02.951556 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:02.951458    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-535bf7ee-g4g7k"
Jul 26 18:20:03.083314 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:03.083286    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-535bf7ee-g4g7k"]
Jul 26 18:20:03.084883 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:20:03.084855    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc109f946_9e81_495e_a102_0061279e3da6.slice/crio-61c1a8a3d244da1d8ee914bf8bb96055167abfb140ee41ad4e9e56acca5c1905 WatchSource:0}: Error finding container 61c1a8a3d244da1d8ee914bf8bb96055167abfb140ee41ad4e9e56acca5c1905: Status 404 returned error can't find the container with id 61c1a8a3d244da1d8ee914bf8bb96055167abfb140ee41ad4e9e56acca5c1905
Jul 26 18:20:03.525732 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:03.525701    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-535bf7ee-g4g7k" event={"ID":"c109f946-9e81-495e-a102-0061279e3da6","Type":"ContainerStarted","Data":"bc1dfbea6e92d9cbd4d5882b8c800fbd726b5ebcbd12f97c8cb279275cd83f68"}
Jul 26 18:20:03.525916 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:03.525755    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-535bf7ee-g4g7k" event={"ID":"c109f946-9e81-495e-a102-0061279e3da6","Type":"ContainerStarted","Data":"61c1a8a3d244da1d8ee914bf8bb96055167abfb140ee41ad4e9e56acca5c1905"}
Jul 26 18:20:03.542078 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:03.542025    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-535bf7ee-g4g7k" podStartSLOduration=1.5420017320000001 podStartE2EDuration="1.542001732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:20:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:20:03.540902694 +0000 UTC m=+1860.193093551" watchObservedRunningTime="2026-07-26 18:20:03.542001732 +0000 UTC m=+1860.194192589"
Jul 26 18:20:04.532459 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:04.532423    2570 generic.go:356] "Generic (PLEG): container finished" podID="c109f946-9e81-495e-a102-0061279e3da6" containerID="bc1dfbea6e92d9cbd4d5882b8c800fbd726b5ebcbd12f97c8cb279275cd83f68" exitCode=0
Jul 26 18:20:04.532922 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:04.532503    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-535bf7ee-g4g7k" event={"ID":"c109f946-9e81-495e-a102-0061279e3da6","Type":"ContainerDied","Data":"bc1dfbea6e92d9cbd4d5882b8c800fbd726b5ebcbd12f97c8cb279275cd83f68"}
Jul 26 18:20:05.678264 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:05.678235    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-535bf7ee-g4g7k"
Jul 26 18:20:05.788525 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:05.788489    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c109f946-9e81-495e-a102-0061279e3da6-maas-api-service-ca\") pod \"c109f946-9e81-495e-a102-0061279e3da6\" (UID: \"c109f946-9e81-495e-a102-0061279e3da6\") "
Jul 26 18:20:05.788917 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:05.788851    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c109f946-9e81-495e-a102-0061279e3da6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c109f946-9e81-495e-a102-0061279e3da6" (UID: "c109f946-9e81-495e-a102-0061279e3da6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:20:05.889584 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:05.889548    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c109f946-9e81-495e-a102-0061279e3da6-maas-api-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:20:06.544947 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:06.544912    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-535bf7ee-g4g7k"
Jul 26 18:20:06.545104 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:06.544911    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-535bf7ee-g4g7k" event={"ID":"c109f946-9e81-495e-a102-0061279e3da6","Type":"ContainerDied","Data":"61c1a8a3d244da1d8ee914bf8bb96055167abfb140ee41ad4e9e56acca5c1905"}
Jul 26 18:20:06.545104 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:06.545068    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61c1a8a3d244da1d8ee914bf8bb96055167abfb140ee41ad4e9e56acca5c1905"
Jul 26 18:20:13.181830 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.181796    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv"]
Jul 26 18:20:13.182428 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.182104    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv" podUID="7f64bf52-840f-49b5-9016-55d214983935" containerName="maas-api" containerID="cri-o://c64f006b893072ec86f02ff3f743fd675e9ddd280eca13e9a90354fe877ee515" gracePeriod=30
Jul 26 18:20:13.224626 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.224591    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk"]
Jul 26 18:20:13.224887 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.224859    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk" podUID="55768422-a4b0-4260-a630-1918eab0030d" containerName="payload-processing" containerID="cri-o://b91fa4d33a3a7253f642733af463728a8cddcf7549cb62392de12508995e911b" gracePeriod=30
Jul 26 18:20:13.231635 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.231608    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272"]
Jul 26 18:20:13.231867 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.231829    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272" podUID="9a3318fa-29b2-4ea8-8a32-ff9f28457492" containerName="payload-pre-processing" containerID="cri-o://e32e7dc26b85870db77cb5c9ec545dd16b56ab19ea4328f592e200f3d010e4c0" gracePeriod=30
Jul 26 18:20:13.495353 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.495326    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv"
Jul 26 18:20:13.498087 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.498065    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk"
Jul 26 18:20:13.531529 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.531507    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272"
Jul 26 18:20:13.558033 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.557992    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f64bf52-840f-49b5-9016-55d214983935-maas-api-tls\") pod \"7f64bf52-840f-49b5-9016-55d214983935\" (UID: \"7f64bf52-840f-49b5-9016-55d214983935\") "
Jul 26 18:20:13.558186 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.558073    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trvmc\" (UniqueName: \"kubernetes.io/projected/55768422-a4b0-4260-a630-1918eab0030d-kube-api-access-trvmc\") pod \"55768422-a4b0-4260-a630-1918eab0030d\" (UID: \"55768422-a4b0-4260-a630-1918eab0030d\") "
Jul 26 18:20:13.558186 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.558170    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f64bf52-840f-49b5-9016-55d214983935-openshift-service-ca\") pod \"7f64bf52-840f-49b5-9016-55d214983935\" (UID: \"7f64bf52-840f-49b5-9016-55d214983935\") "
Jul 26 18:20:13.558271 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.558205    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8gk8\" (UniqueName: \"kubernetes.io/projected/7f64bf52-840f-49b5-9016-55d214983935-kube-api-access-q8gk8\") pod \"7f64bf52-840f-49b5-9016-55d214983935\" (UID: \"7f64bf52-840f-49b5-9016-55d214983935\") "
Jul 26 18:20:13.558271 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.558256    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55768422-a4b0-4260-a630-1918eab0030d-plugins-config-volume\") pod \"55768422-a4b0-4260-a630-1918eab0030d\" (UID: \"55768422-a4b0-4260-a630-1918eab0030d\") "
Jul 26 18:20:13.558651 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.558544    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f64bf52-840f-49b5-9016-55d214983935-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7f64bf52-840f-49b5-9016-55d214983935" (UID: "7f64bf52-840f-49b5-9016-55d214983935"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:20:13.558763 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.558742    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55768422-a4b0-4260-a630-1918eab0030d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "55768422-a4b0-4260-a630-1918eab0030d" (UID: "55768422-a4b0-4260-a630-1918eab0030d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:20:13.560293 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.560268    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f64bf52-840f-49b5-9016-55d214983935-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7f64bf52-840f-49b5-9016-55d214983935" (UID: "7f64bf52-840f-49b5-9016-55d214983935"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:20:13.560397 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.560318    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55768422-a4b0-4260-a630-1918eab0030d-kube-api-access-trvmc" (OuterVolumeSpecName: "kube-api-access-trvmc") pod "55768422-a4b0-4260-a630-1918eab0030d" (UID: "55768422-a4b0-4260-a630-1918eab0030d"). InnerVolumeSpecName "kube-api-access-trvmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:20:13.560397 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.560328    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f64bf52-840f-49b5-9016-55d214983935-kube-api-access-q8gk8" (OuterVolumeSpecName: "kube-api-access-q8gk8") pod "7f64bf52-840f-49b5-9016-55d214983935" (UID: "7f64bf52-840f-49b5-9016-55d214983935"). InnerVolumeSpecName "kube-api-access-q8gk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:20:13.584189 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.584163    2570 generic.go:356] "Generic (PLEG): container finished" podID="7f64bf52-840f-49b5-9016-55d214983935" containerID="c64f006b893072ec86f02ff3f743fd675e9ddd280eca13e9a90354fe877ee515" exitCode=0
Jul 26 18:20:13.584297 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.584205    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv"
Jul 26 18:20:13.584297 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.584244    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv" event={"ID":"7f64bf52-840f-49b5-9016-55d214983935","Type":"ContainerDied","Data":"c64f006b893072ec86f02ff3f743fd675e9ddd280eca13e9a90354fe877ee515"}
Jul 26 18:20:13.584382 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.584297    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv" event={"ID":"7f64bf52-840f-49b5-9016-55d214983935","Type":"ContainerDied","Data":"882e24ebe24ce0c9b6b1a2b992739538eb53af59356829a757e7abc85c1e974c"}
Jul 26 18:20:13.584382 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.584324    2570 scope.go:117] "RemoveContainer" containerID="c64f006b893072ec86f02ff3f743fd675e9ddd280eca13e9a90354fe877ee515"
Jul 26 18:20:13.585702 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.585683    2570 generic.go:356] "Generic (PLEG): container finished" podID="55768422-a4b0-4260-a630-1918eab0030d" containerID="b91fa4d33a3a7253f642733af463728a8cddcf7549cb62392de12508995e911b" exitCode=0
Jul 26 18:20:13.585799 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.585730    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk"
Jul 26 18:20:13.585799 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.585768    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk" event={"ID":"55768422-a4b0-4260-a630-1918eab0030d","Type":"ContainerDied","Data":"b91fa4d33a3a7253f642733af463728a8cddcf7549cb62392de12508995e911b"}
Jul 26 18:20:13.585916 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.585812    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk" event={"ID":"55768422-a4b0-4260-a630-1918eab0030d","Type":"ContainerDied","Data":"f6fe59993411893c59d43f6be77dcf033e532784f044abd40bf989b955973225"}
Jul 26 18:20:13.587095 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.587073    2570 generic.go:356] "Generic (PLEG): container finished" podID="9a3318fa-29b2-4ea8-8a32-ff9f28457492" containerID="e32e7dc26b85870db77cb5c9ec545dd16b56ab19ea4328f592e200f3d010e4c0" exitCode=0
Jul 26 18:20:13.587222 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.587103    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272" event={"ID":"9a3318fa-29b2-4ea8-8a32-ff9f28457492","Type":"ContainerDied","Data":"e32e7dc26b85870db77cb5c9ec545dd16b56ab19ea4328f592e200f3d010e4c0"}
Jul 26 18:20:13.587222 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.587146    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272"
Jul 26 18:20:13.587222 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.587155    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272" event={"ID":"9a3318fa-29b2-4ea8-8a32-ff9f28457492","Type":"ContainerDied","Data":"3ffdff2141a414bc2ee9b34b9b1028d77efd2fc4276b9dd200d37c2b580c7901"}
Jul 26 18:20:13.593790 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.593772    2570 scope.go:117] "RemoveContainer" containerID="c64f006b893072ec86f02ff3f743fd675e9ddd280eca13e9a90354fe877ee515"
Jul 26 18:20:13.594171 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:20:13.594059    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c64f006b893072ec86f02ff3f743fd675e9ddd280eca13e9a90354fe877ee515\": container with ID starting with c64f006b893072ec86f02ff3f743fd675e9ddd280eca13e9a90354fe877ee515 not found: ID does not exist" containerID="c64f006b893072ec86f02ff3f743fd675e9ddd280eca13e9a90354fe877ee515"
Jul 26 18:20:13.594171 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.594098    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c64f006b893072ec86f02ff3f743fd675e9ddd280eca13e9a90354fe877ee515"} err="failed to get container status \"c64f006b893072ec86f02ff3f743fd675e9ddd280eca13e9a90354fe877ee515\": rpc error: code = NotFound desc = could not find container \"c64f006b893072ec86f02ff3f743fd675e9ddd280eca13e9a90354fe877ee515\": container with ID starting with c64f006b893072ec86f02ff3f743fd675e9ddd280eca13e9a90354fe877ee515 not found: ID does not exist"
Jul 26 18:20:13.594171 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.594153    2570 scope.go:117] "RemoveContainer" containerID="b91fa4d33a3a7253f642733af463728a8cddcf7549cb62392de12508995e911b"
Jul 26 18:20:13.602546 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.602532    2570 scope.go:117] "RemoveContainer" containerID="b91fa4d33a3a7253f642733af463728a8cddcf7549cb62392de12508995e911b"
Jul 26 18:20:13.602789 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:20:13.602772    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b91fa4d33a3a7253f642733af463728a8cddcf7549cb62392de12508995e911b\": container with ID starting with b91fa4d33a3a7253f642733af463728a8cddcf7549cb62392de12508995e911b not found: ID does not exist" containerID="b91fa4d33a3a7253f642733af463728a8cddcf7549cb62392de12508995e911b"
Jul 26 18:20:13.602833 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.602797    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b91fa4d33a3a7253f642733af463728a8cddcf7549cb62392de12508995e911b"} err="failed to get container status \"b91fa4d33a3a7253f642733af463728a8cddcf7549cb62392de12508995e911b\": rpc error: code = NotFound desc = could not find container \"b91fa4d33a3a7253f642733af463728a8cddcf7549cb62392de12508995e911b\": container with ID starting with b91fa4d33a3a7253f642733af463728a8cddcf7549cb62392de12508995e911b not found: ID does not exist"
Jul 26 18:20:13.602833 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.602814    2570 scope.go:117] "RemoveContainer" containerID="e32e7dc26b85870db77cb5c9ec545dd16b56ab19ea4328f592e200f3d010e4c0"
Jul 26 18:20:13.610541 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.610526    2570 scope.go:117] "RemoveContainer" containerID="e32e7dc26b85870db77cb5c9ec545dd16b56ab19ea4328f592e200f3d010e4c0"
Jul 26 18:20:13.610767 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:20:13.610742    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e32e7dc26b85870db77cb5c9ec545dd16b56ab19ea4328f592e200f3d010e4c0\": container with ID starting with e32e7dc26b85870db77cb5c9ec545dd16b56ab19ea4328f592e200f3d010e4c0 not found: ID does not exist" containerID="e32e7dc26b85870db77cb5c9ec545dd16b56ab19ea4328f592e200f3d010e4c0"
Jul 26 18:20:13.610826 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.610776    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e32e7dc26b85870db77cb5c9ec545dd16b56ab19ea4328f592e200f3d010e4c0"} err="failed to get container status \"e32e7dc26b85870db77cb5c9ec545dd16b56ab19ea4328f592e200f3d010e4c0\": rpc error: code = NotFound desc = could not find container \"e32e7dc26b85870db77cb5c9ec545dd16b56ab19ea4328f592e200f3d010e4c0\": container with ID starting with e32e7dc26b85870db77cb5c9ec545dd16b56ab19ea4328f592e200f3d010e4c0 not found: ID does not exist"
Jul 26 18:20:13.612231 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.612211    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv"]
Jul 26 18:20:13.614561 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.614540    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-535bf7ee-5b6c7f9878-khpnv"]
Jul 26 18:20:13.624867 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.624848    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk"]
Jul 26 18:20:13.629811 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.629790    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-535bf7ee-865fb7cdd6-mmpgk"]
Jul 26 18:20:13.658905 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.658883    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7gcl6\" (UniqueName: \"kubernetes.io/projected/9a3318fa-29b2-4ea8-8a32-ff9f28457492-kube-api-access-7gcl6\") pod \"9a3318fa-29b2-4ea8-8a32-ff9f28457492\" (UID: \"9a3318fa-29b2-4ea8-8a32-ff9f28457492\") "
Jul 26 18:20:13.658991 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.658968    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a3318fa-29b2-4ea8-8a32-ff9f28457492-plugins-config-volume\") pod \"9a3318fa-29b2-4ea8-8a32-ff9f28457492\" (UID: \"9a3318fa-29b2-4ea8-8a32-ff9f28457492\") "
Jul 26 18:20:13.659238 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.659220    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55768422-a4b0-4260-a630-1918eab0030d-plugins-config-volume\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:20:13.659286 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.659245    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f64bf52-840f-49b5-9016-55d214983935-maas-api-tls\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:20:13.659286 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.659259    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trvmc\" (UniqueName: \"kubernetes.io/projected/55768422-a4b0-4260-a630-1918eab0030d-kube-api-access-trvmc\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:20:13.659286 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.659274    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f64bf52-840f-49b5-9016-55d214983935-openshift-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:20:13.659399 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.659290    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8gk8\" (UniqueName: \"kubernetes.io/projected/7f64bf52-840f-49b5-9016-55d214983935-kube-api-access-q8gk8\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:20:13.659399 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.659368    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a3318fa-29b2-4ea8-8a32-ff9f28457492-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9a3318fa-29b2-4ea8-8a32-ff9f28457492" (UID: "9a3318fa-29b2-4ea8-8a32-ff9f28457492"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:20:13.660801 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.660780    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a3318fa-29b2-4ea8-8a32-ff9f28457492-kube-api-access-7gcl6" (OuterVolumeSpecName: "kube-api-access-7gcl6") pod "9a3318fa-29b2-4ea8-8a32-ff9f28457492" (UID: "9a3318fa-29b2-4ea8-8a32-ff9f28457492"). InnerVolumeSpecName "kube-api-access-7gcl6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:20:13.760473 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.760446    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a3318fa-29b2-4ea8-8a32-ff9f28457492-plugins-config-volume\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:20:13.760473 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.760471    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gcl6\" (UniqueName: \"kubernetes.io/projected/9a3318fa-29b2-4ea8-8a32-ff9f28457492-kube-api-access-7gcl6\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:20:13.839860 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.839826    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55768422-a4b0-4260-a630-1918eab0030d" path="/var/lib/kubelet/pods/55768422-a4b0-4260-a630-1918eab0030d/volumes"
Jul 26 18:20:13.840235 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.840221    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f64bf52-840f-49b5-9016-55d214983935" path="/var/lib/kubelet/pods/7f64bf52-840f-49b5-9016-55d214983935/volumes"
Jul 26 18:20:13.903149 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.903105    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272"]
Jul 26 18:20:13.906434 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:13.906409    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-535bf7ee-54b565c7b8-rp272"]
Jul 26 18:20:15.838805 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:15.838768    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a3318fa-29b2-4ea8-8a32-ff9f28457492" path="/var/lib/kubelet/pods/9a3318fa-29b2-4ea8-8a32-ff9f28457492/volumes"
Jul 26 18:20:17.697877 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:17.697841    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-535bf7ee-g4g7k"]
Jul 26 18:20:17.700895 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:17.700869    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-535bf7ee-g4g7k"]
Jul 26 18:20:17.839050 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:20:17.839016    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c109f946-9e81-495e-a102-0061279e3da6" path="/var/lib/kubelet/pods/c109f946-9e81-495e-a102-0061279e3da6/volumes"
Jul 26 18:21:58.276316 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.276088    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4"]
Jul 26 18:21:58.277474 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.277442    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55768422-a4b0-4260-a630-1918eab0030d" containerName="payload-processing"
Jul 26 18:21:58.277474 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.277473    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="55768422-a4b0-4260-a630-1918eab0030d" containerName="payload-processing"
Jul 26 18:21:58.277633 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.277491    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f64bf52-840f-49b5-9016-55d214983935" containerName="maas-api"
Jul 26 18:21:58.277633 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.277502    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f64bf52-840f-49b5-9016-55d214983935" containerName="maas-api"
Jul 26 18:21:58.277633 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.277530    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c109f946-9e81-495e-a102-0061279e3da6" containerName="revoke-keys"
Jul 26 18:21:58.277633 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.277539    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c109f946-9e81-495e-a102-0061279e3da6" containerName="revoke-keys"
Jul 26 18:21:58.277633 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.277559    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a3318fa-29b2-4ea8-8a32-ff9f28457492" containerName="payload-pre-processing"
Jul 26 18:21:58.277633 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.277568    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a3318fa-29b2-4ea8-8a32-ff9f28457492" containerName="payload-pre-processing"
Jul 26 18:21:58.277828 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.277678    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f64bf52-840f-49b5-9016-55d214983935" containerName="maas-api"
Jul 26 18:21:58.277828 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.277694    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a3318fa-29b2-4ea8-8a32-ff9f28457492" containerName="payload-pre-processing"
Jul 26 18:21:58.277828 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.277703    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="55768422-a4b0-4260-a630-1918eab0030d" containerName="payload-processing"
Jul 26 18:21:58.277828 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.277716    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c109f946-9e81-495e-a102-0061279e3da6" containerName="revoke-keys"
Jul 26 18:21:58.280885 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.280868    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4"
Jul 26 18:21:58.283063 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.283039    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-d700fe05-serving-cert\""
Jul 26 18:21:58.290104 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.290079    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4"]
Jul 26 18:21:58.369883 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.369854    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhn7b\" (UniqueName: \"kubernetes.io/projected/d05a74cc-01db-4d99-9e5c-ae48a95597de-kube-api-access-hhn7b\") pod \"maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4\" (UID: \"d05a74cc-01db-4d99-9e5c-ae48a95597de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4"
Jul 26 18:21:58.370078 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.369896    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d05a74cc-01db-4d99-9e5c-ae48a95597de-maas-api-tls\") pod \"maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4\" (UID: \"d05a74cc-01db-4d99-9e5c-ae48a95597de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4"
Jul 26 18:21:58.370078 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.369932    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d05a74cc-01db-4d99-9e5c-ae48a95597de-openshift-service-ca\") pod \"maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4\" (UID: \"d05a74cc-01db-4d99-9e5c-ae48a95597de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4"
Jul 26 18:21:58.471008 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.470969    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhn7b\" (UniqueName: \"kubernetes.io/projected/d05a74cc-01db-4d99-9e5c-ae48a95597de-kube-api-access-hhn7b\") pod \"maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4\" (UID: \"d05a74cc-01db-4d99-9e5c-ae48a95597de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4"
Jul 26 18:21:58.471235 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.471017    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d05a74cc-01db-4d99-9e5c-ae48a95597de-maas-api-tls\") pod \"maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4\" (UID: \"d05a74cc-01db-4d99-9e5c-ae48a95597de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4"
Jul 26 18:21:58.471235 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.471050    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d05a74cc-01db-4d99-9e5c-ae48a95597de-openshift-service-ca\") pod \"maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4\" (UID: \"d05a74cc-01db-4d99-9e5c-ae48a95597de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4"
Jul 26 18:21:58.471751 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.471726    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d05a74cc-01db-4d99-9e5c-ae48a95597de-openshift-service-ca\") pod \"maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4\" (UID: \"d05a74cc-01db-4d99-9e5c-ae48a95597de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4"
Jul 26 18:21:58.473713 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.473694    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d05a74cc-01db-4d99-9e5c-ae48a95597de-maas-api-tls\") pod \"maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4\" (UID: \"d05a74cc-01db-4d99-9e5c-ae48a95597de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4"
Jul 26 18:21:58.480981 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.480954    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhn7b\" (UniqueName: \"kubernetes.io/projected/d05a74cc-01db-4d99-9e5c-ae48a95597de-kube-api-access-hhn7b\") pod \"maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4\" (UID: \"d05a74cc-01db-4d99-9e5c-ae48a95597de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4"
Jul 26 18:21:58.594032 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.593931    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4"
Jul 26 18:21:58.736732 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.736693    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4"]
Jul 26 18:21:58.738292 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:21:58.738254    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd05a74cc_01db_4d99_9e5c_ae48a95597de.slice/crio-a8540eea831f8a255249935ef39116769ad0ec0b035584a432160d5dcd9e7b95 WatchSource:0}: Error finding container a8540eea831f8a255249935ef39116769ad0ec0b035584a432160d5dcd9e7b95: Status 404 returned error can't find the container with id a8540eea831f8a255249935ef39116769ad0ec0b035584a432160d5dcd9e7b95
Jul 26 18:21:58.953898 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.953815    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm"]
Jul 26 18:21:58.958386 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.958365    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm"
Jul 26 18:21:58.960883 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.960863    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-d700fe05\""
Jul 26 18:21:58.960983 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.960884    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-d700fe05-dockercfg-hqlk9\""
Jul 26 18:21:58.972911 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.972861    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm"]
Jul 26 18:21:58.979428 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.979404    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs"]
Jul 26 18:21:58.983291 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.983271    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs"
Jul 26 18:21:58.991620 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:58.991596    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs"]
Jul 26 18:21:59.078050 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:59.078010    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v52hg\" (UniqueName: \"kubernetes.io/projected/7348fd2f-eba6-4334-9555-483772fe1a22-kube-api-access-v52hg\") pod \"payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm\" (UID: \"7348fd2f-eba6-4334-9555-483772fe1a22\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm"
Jul 26 18:21:59.078241 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:59.078057    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq6h5\" (UniqueName: \"kubernetes.io/projected/533a155f-73fd-485d-adc7-a122d5415104-kube-api-access-xq6h5\") pod \"payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs\" (UID: \"533a155f-73fd-485d-adc7-a122d5415104\") " pod="openshift-ingress/payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs"
Jul 26 18:21:59.078241 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:59.078200    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/533a155f-73fd-485d-adc7-a122d5415104-plugins-config-volume\") pod \"payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs\" (UID: \"533a155f-73fd-485d-adc7-a122d5415104\") " pod="openshift-ingress/payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs"
Jul 26 18:21:59.078241 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:59.078225    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7348fd2f-eba6-4334-9555-483772fe1a22-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm\" (UID: \"7348fd2f-eba6-4334-9555-483772fe1a22\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm"
Jul 26 18:21:59.178735 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:59.178702    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v52hg\" (UniqueName: \"kubernetes.io/projected/7348fd2f-eba6-4334-9555-483772fe1a22-kube-api-access-v52hg\") pod \"payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm\" (UID: \"7348fd2f-eba6-4334-9555-483772fe1a22\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm"
Jul 26 18:21:59.178882 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:59.178750    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq6h5\" (UniqueName: \"kubernetes.io/projected/533a155f-73fd-485d-adc7-a122d5415104-kube-api-access-xq6h5\") pod \"payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs\" (UID: \"533a155f-73fd-485d-adc7-a122d5415104\") " pod="openshift-ingress/payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs"
Jul 26 18:21:59.178882 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:59.178840    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/533a155f-73fd-485d-adc7-a122d5415104-plugins-config-volume\") pod \"payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs\" (UID: \"533a155f-73fd-485d-adc7-a122d5415104\") " pod="openshift-ingress/payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs"
Jul 26 18:21:59.178882 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:59.178867    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7348fd2f-eba6-4334-9555-483772fe1a22-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm\" (UID: \"7348fd2f-eba6-4334-9555-483772fe1a22\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm"
Jul 26 18:21:59.179737 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:59.179707    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/533a155f-73fd-485d-adc7-a122d5415104-plugins-config-volume\") pod \"payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs\" (UID: \"533a155f-73fd-485d-adc7-a122d5415104\") " pod="openshift-ingress/payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs"
Jul 26 18:21:59.179859 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:59.179762    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7348fd2f-eba6-4334-9555-483772fe1a22-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm\" (UID: \"7348fd2f-eba6-4334-9555-483772fe1a22\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm"
Jul 26 18:21:59.179956 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:59.179933    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4" event={"ID":"d05a74cc-01db-4d99-9e5c-ae48a95597de","Type":"ContainerStarted","Data":"b8bbb407590d2f0e91a1e36b3b88c710387c9327a7d0f6239fe90102f2243d25"}
Jul 26 18:21:59.180019 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:59.179992    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4"
Jul 26 18:21:59.180077 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:59.180018    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4" event={"ID":"d05a74cc-01db-4d99-9e5c-ae48a95597de","Type":"ContainerStarted","Data":"a8540eea831f8a255249935ef39116769ad0ec0b035584a432160d5dcd9e7b95"}
Jul 26 18:21:59.186893 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:59.186870    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v52hg\" (UniqueName: \"kubernetes.io/projected/7348fd2f-eba6-4334-9555-483772fe1a22-kube-api-access-v52hg\") pod \"payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm\" (UID: \"7348fd2f-eba6-4334-9555-483772fe1a22\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm"
Jul 26 18:21:59.187185 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:59.187165    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq6h5\" (UniqueName: \"kubernetes.io/projected/533a155f-73fd-485d-adc7-a122d5415104-kube-api-access-xq6h5\") pod \"payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs\" (UID: \"533a155f-73fd-485d-adc7-a122d5415104\") " pod="openshift-ingress/payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs"
Jul 26 18:21:59.196046 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:59.196006    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4" podStartSLOduration=0.829925939 podStartE2EDuration="1.195993597s" totalImagesPullingTime="366.067658ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:21:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:21:59.194194458 +0000 UTC m=+1975.846385318" watchObservedRunningTime="2026-07-26 18:21:59.195993597 +0000 UTC m=+1975.848184455"
Jul 26 18:21:59.270680 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:59.270650    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm"
Jul 26 18:21:59.295021 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:59.294994    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs"
Jul 26 18:21:59.420578 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:59.420548    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm"]
Jul 26 18:21:59.422846 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:21:59.422807    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7348fd2f_eba6_4334_9555_483772fe1a22.slice/crio-f354d4812328d5854ab4c1e0047ef967ca097fa44bf60280a22bd289fb4ae676 WatchSource:0}: Error finding container f354d4812328d5854ab4c1e0047ef967ca097fa44bf60280a22bd289fb4ae676: Status 404 returned error can't find the container with id f354d4812328d5854ab4c1e0047ef967ca097fa44bf60280a22bd289fb4ae676
Jul 26 18:21:59.448928 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:21:59.448902    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs"]
Jul 26 18:21:59.462444 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:21:59.462419    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod533a155f_73fd_485d_adc7_a122d5415104.slice/crio-9f024bf50b7c5230c85dac6dd40dbe3c7df1ab2ce96145afac538d4260f91b8c WatchSource:0}: Error finding container 9f024bf50b7c5230c85dac6dd40dbe3c7df1ab2ce96145afac538d4260f91b8c: Status 404 returned error can't find the container with id 9f024bf50b7c5230c85dac6dd40dbe3c7df1ab2ce96145afac538d4260f91b8c
Jul 26 18:22:00.186628 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:00.186592    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs" event={"ID":"533a155f-73fd-485d-adc7-a122d5415104","Type":"ContainerStarted","Data":"306d8a379e6c8cf7329801a6f6f46b85d6833e913db9d820fd578ad8b9d672ab"}
Jul 26 18:22:00.186848 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:00.186649    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs" event={"ID":"533a155f-73fd-485d-adc7-a122d5415104","Type":"ContainerStarted","Data":"9f024bf50b7c5230c85dac6dd40dbe3c7df1ab2ce96145afac538d4260f91b8c"}
Jul 26 18:22:00.186848 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:00.186675    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs"
Jul 26 18:22:00.188108 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:00.188080    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm" event={"ID":"7348fd2f-eba6-4334-9555-483772fe1a22","Type":"ContainerStarted","Data":"86117164cc6e9c1345bb5c7d5323845b649a5e170938d6bd947c8f675c884562"}
Jul 26 18:22:00.188237 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:00.188142    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm" event={"ID":"7348fd2f-eba6-4334-9555-483772fe1a22","Type":"ContainerStarted","Data":"f354d4812328d5854ab4c1e0047ef967ca097fa44bf60280a22bd289fb4ae676"}
Jul 26 18:22:00.188296 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:00.188280    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm"
Jul 26 18:22:00.206463 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:00.206415    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs" podStartSLOduration=2.206400289 podStartE2EDuration="2.206400289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:21:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:22:00.20396974 +0000 UTC m=+1976.856160597" watchObservedRunningTime="2026-07-26 18:22:00.206400289 +0000 UTC m=+1976.858591147"
Jul 26 18:22:00.223080 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:00.223030    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm" podStartSLOduration=2.223015956 podStartE2EDuration="2.223015956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:21:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:22:00.221360219 +0000 UTC m=+1976.873551078" watchObservedRunningTime="2026-07-26 18:22:00.223015956 +0000 UTC m=+1976.875206814"
Jul 26 18:22:05.193902 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:05.193873    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4"
Jul 26 18:22:11.195480 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:11.195447    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm"
Jul 26 18:22:11.195891 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:11.195518    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs"
Jul 26 18:22:20.040656 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.040622    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4"]
Jul 26 18:22:20.041108 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.040815    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4" podUID="d05a74cc-01db-4d99-9e5c-ae48a95597de" containerName="maas-api" containerID="cri-o://b8bbb407590d2f0e91a1e36b3b88c710387c9327a7d0f6239fe90102f2243d25" gracePeriod=30
Jul 26 18:22:20.098469 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.098434    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs"]
Jul 26 18:22:20.098765 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.098715    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs" podUID="533a155f-73fd-485d-adc7-a122d5415104" containerName="payload-processing" containerID="cri-o://306d8a379e6c8cf7329801a6f6f46b85d6833e913db9d820fd578ad8b9d672ab" gracePeriod=30
Jul 26 18:22:20.106583 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.106554    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm"]
Jul 26 18:22:20.106855 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.106795    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm" podUID="7348fd2f-eba6-4334-9555-483772fe1a22" containerName="payload-pre-processing" containerID="cri-o://86117164cc6e9c1345bb5c7d5323845b649a5e170938d6bd947c8f675c884562" gracePeriod=30
Jul 26 18:22:20.189124 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.189076    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4" podUID="d05a74cc-01db-4d99-9e5c-ae48a95597de" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.70:8443/health\": dial tcp 10.132.0.70:8443: connect: connection refused"
Jul 26 18:22:20.300469 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.300388    2570 generic.go:356] "Generic (PLEG): container finished" podID="533a155f-73fd-485d-adc7-a122d5415104" containerID="306d8a379e6c8cf7329801a6f6f46b85d6833e913db9d820fd578ad8b9d672ab" exitCode=0
Jul 26 18:22:20.300604 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.300457    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs" event={"ID":"533a155f-73fd-485d-adc7-a122d5415104","Type":"ContainerDied","Data":"306d8a379e6c8cf7329801a6f6f46b85d6833e913db9d820fd578ad8b9d672ab"}
Jul 26 18:22:20.302031 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.302000    2570 generic.go:356] "Generic (PLEG): container finished" podID="d05a74cc-01db-4d99-9e5c-ae48a95597de" containerID="b8bbb407590d2f0e91a1e36b3b88c710387c9327a7d0f6239fe90102f2243d25" exitCode=0
Jul 26 18:22:20.302262 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.302063    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4" event={"ID":"d05a74cc-01db-4d99-9e5c-ae48a95597de","Type":"ContainerDied","Data":"b8bbb407590d2f0e91a1e36b3b88c710387c9327a7d0f6239fe90102f2243d25"}
Jul 26 18:22:20.303871 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.303846    2570 generic.go:356] "Generic (PLEG): container finished" podID="7348fd2f-eba6-4334-9555-483772fe1a22" containerID="86117164cc6e9c1345bb5c7d5323845b649a5e170938d6bd947c8f675c884562" exitCode=0
Jul 26 18:22:20.304148 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.303899    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm" event={"ID":"7348fd2f-eba6-4334-9555-483772fe1a22","Type":"ContainerDied","Data":"86117164cc6e9c1345bb5c7d5323845b649a5e170938d6bd947c8f675c884562"}
Jul 26 18:22:20.378431 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.378408    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs"
Jul 26 18:22:20.382247 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.382225    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4"
Jul 26 18:22:20.398975 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.398946    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm"
Jul 26 18:22:20.575501 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.575414    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/533a155f-73fd-485d-adc7-a122d5415104-plugins-config-volume\") pod \"533a155f-73fd-485d-adc7-a122d5415104\" (UID: \"533a155f-73fd-485d-adc7-a122d5415104\") "
Jul 26 18:22:20.575501 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.575449    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7348fd2f-eba6-4334-9555-483772fe1a22-plugins-config-volume\") pod \"7348fd2f-eba6-4334-9555-483772fe1a22\" (UID: \"7348fd2f-eba6-4334-9555-483772fe1a22\") "
Jul 26 18:22:20.575501 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.575487    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v52hg\" (UniqueName: \"kubernetes.io/projected/7348fd2f-eba6-4334-9555-483772fe1a22-kube-api-access-v52hg\") pod \"7348fd2f-eba6-4334-9555-483772fe1a22\" (UID: \"7348fd2f-eba6-4334-9555-483772fe1a22\") "
Jul 26 18:22:20.575731 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.575697    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhn7b\" (UniqueName: \"kubernetes.io/projected/d05a74cc-01db-4d99-9e5c-ae48a95597de-kube-api-access-hhn7b\") pod \"d05a74cc-01db-4d99-9e5c-ae48a95597de\" (UID: \"d05a74cc-01db-4d99-9e5c-ae48a95597de\") "
Jul 26 18:22:20.575792 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.575776    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d05a74cc-01db-4d99-9e5c-ae48a95597de-maas-api-tls\") pod \"d05a74cc-01db-4d99-9e5c-ae48a95597de\" (UID: \"d05a74cc-01db-4d99-9e5c-ae48a95597de\") "
Jul 26 18:22:20.575899 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.575881    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xq6h5\" (UniqueName: \"kubernetes.io/projected/533a155f-73fd-485d-adc7-a122d5415104-kube-api-access-xq6h5\") pod \"533a155f-73fd-485d-adc7-a122d5415104\" (UID: \"533a155f-73fd-485d-adc7-a122d5415104\") "
Jul 26 18:22:20.575899 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.575886    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7348fd2f-eba6-4334-9555-483772fe1a22-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7348fd2f-eba6-4334-9555-483772fe1a22" (UID: "7348fd2f-eba6-4334-9555-483772fe1a22"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:22:20.576015 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.575889    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/533a155f-73fd-485d-adc7-a122d5415104-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "533a155f-73fd-485d-adc7-a122d5415104" (UID: "533a155f-73fd-485d-adc7-a122d5415104"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:22:20.576015 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.575919    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d05a74cc-01db-4d99-9e5c-ae48a95597de-openshift-service-ca\") pod \"d05a74cc-01db-4d99-9e5c-ae48a95597de\" (UID: \"d05a74cc-01db-4d99-9e5c-ae48a95597de\") "
Jul 26 18:22:20.576281 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.576261    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/533a155f-73fd-485d-adc7-a122d5415104-plugins-config-volume\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:22:20.576386 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.576285    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7348fd2f-eba6-4334-9555-483772fe1a22-plugins-config-volume\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:22:20.576450 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.576426    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d05a74cc-01db-4d99-9e5c-ae48a95597de-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d05a74cc-01db-4d99-9e5c-ae48a95597de" (UID: "d05a74cc-01db-4d99-9e5c-ae48a95597de"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:22:20.577880 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.577846    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d05a74cc-01db-4d99-9e5c-ae48a95597de-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d05a74cc-01db-4d99-9e5c-ae48a95597de" (UID: "d05a74cc-01db-4d99-9e5c-ae48a95597de"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:22:20.577961 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.577920    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d05a74cc-01db-4d99-9e5c-ae48a95597de-kube-api-access-hhn7b" (OuterVolumeSpecName: "kube-api-access-hhn7b") pod "d05a74cc-01db-4d99-9e5c-ae48a95597de" (UID: "d05a74cc-01db-4d99-9e5c-ae48a95597de"). InnerVolumeSpecName "kube-api-access-hhn7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:22:20.577961 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.577949    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7348fd2f-eba6-4334-9555-483772fe1a22-kube-api-access-v52hg" (OuterVolumeSpecName: "kube-api-access-v52hg") pod "7348fd2f-eba6-4334-9555-483772fe1a22" (UID: "7348fd2f-eba6-4334-9555-483772fe1a22"). InnerVolumeSpecName "kube-api-access-v52hg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:22:20.577961 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.577945    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/533a155f-73fd-485d-adc7-a122d5415104-kube-api-access-xq6h5" (OuterVolumeSpecName: "kube-api-access-xq6h5") pod "533a155f-73fd-485d-adc7-a122d5415104" (UID: "533a155f-73fd-485d-adc7-a122d5415104"). InnerVolumeSpecName "kube-api-access-xq6h5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:22:20.677551 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.677513    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xq6h5\" (UniqueName: \"kubernetes.io/projected/533a155f-73fd-485d-adc7-a122d5415104-kube-api-access-xq6h5\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:22:20.677551 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.677543    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d05a74cc-01db-4d99-9e5c-ae48a95597de-openshift-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:22:20.677551 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.677554    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v52hg\" (UniqueName: \"kubernetes.io/projected/7348fd2f-eba6-4334-9555-483772fe1a22-kube-api-access-v52hg\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:22:20.677780 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.677562    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhn7b\" (UniqueName: \"kubernetes.io/projected/d05a74cc-01db-4d99-9e5c-ae48a95597de-kube-api-access-hhn7b\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:22:20.677780 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:20.677572    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d05a74cc-01db-4d99-9e5c-ae48a95597de-maas-api-tls\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:22:21.310717 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:21.310632    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4"
Jul 26 18:22:21.310717 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:21.310637    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4" event={"ID":"d05a74cc-01db-4d99-9e5c-ae48a95597de","Type":"ContainerDied","Data":"a8540eea831f8a255249935ef39116769ad0ec0b035584a432160d5dcd9e7b95"}
Jul 26 18:22:21.310717 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:21.310702    2570 scope.go:117] "RemoveContainer" containerID="b8bbb407590d2f0e91a1e36b3b88c710387c9327a7d0f6239fe90102f2243d25"
Jul 26 18:22:21.316590 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:21.316568    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm"
Jul 26 18:22:21.319523 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:21.316772    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm" event={"ID":"7348fd2f-eba6-4334-9555-483772fe1a22","Type":"ContainerDied","Data":"f354d4812328d5854ab4c1e0047ef967ca097fa44bf60280a22bd289fb4ae676"}
Jul 26 18:22:21.321464 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:21.321442    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs"
Jul 26 18:22:21.321592 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:21.321456    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs" event={"ID":"533a155f-73fd-485d-adc7-a122d5415104","Type":"ContainerDied","Data":"9f024bf50b7c5230c85dac6dd40dbe3c7df1ab2ce96145afac538d4260f91b8c"}
Jul 26 18:22:21.326783 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:21.326760    2570 scope.go:117] "RemoveContainer" containerID="86117164cc6e9c1345bb5c7d5323845b649a5e170938d6bd947c8f675c884562"
Jul 26 18:22:21.335748 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:21.335727    2570 scope.go:117] "RemoveContainer" containerID="306d8a379e6c8cf7329801a6f6f46b85d6833e913db9d820fd578ad8b9d672ab"
Jul 26 18:22:21.343921 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:21.343897    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4"]
Jul 26 18:22:21.347492 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:21.347469    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d700fe05-7d97fd8b97-7pwf4"]
Jul 26 18:22:21.361532 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:21.361508    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs"]
Jul 26 18:22:21.364134 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:21.364099    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d700fe05-7475ffd874-pb2hs"]
Jul 26 18:22:21.375028 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:21.375007    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm"]
Jul 26 18:22:21.379517 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:21.379496    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d700fe05-fcd494f84-klpwm"]
Jul 26 18:22:21.838980 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:21.838949    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="533a155f-73fd-485d-adc7-a122d5415104" path="/var/lib/kubelet/pods/533a155f-73fd-485d-adc7-a122d5415104/volumes"
Jul 26 18:22:21.839361 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:21.839347    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7348fd2f-eba6-4334-9555-483772fe1a22" path="/var/lib/kubelet/pods/7348fd2f-eba6-4334-9555-483772fe1a22/volumes"
Jul 26 18:22:21.839690 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:21.839678    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d05a74cc-01db-4d99-9e5c-ae48a95597de" path="/var/lib/kubelet/pods/d05a74cc-01db-4d99-9e5c-ae48a95597de/volumes"
Jul 26 18:22:54.492812 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:54.492773    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9"]
Jul 26 18:22:54.495443 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:54.493128    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d05a74cc-01db-4d99-9e5c-ae48a95597de" containerName="maas-api"
Jul 26 18:22:54.495443 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:54.493145    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="d05a74cc-01db-4d99-9e5c-ae48a95597de" containerName="maas-api"
Jul 26 18:22:54.495443 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:54.493162    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="533a155f-73fd-485d-adc7-a122d5415104" containerName="payload-processing"
Jul 26 18:22:54.495443 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:54.493168    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="533a155f-73fd-485d-adc7-a122d5415104" containerName="payload-processing"
Jul 26 18:22:54.495443 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:54.493184    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7348fd2f-eba6-4334-9555-483772fe1a22" containerName="payload-pre-processing"
Jul 26 18:22:54.495443 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:54.493190    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7348fd2f-eba6-4334-9555-483772fe1a22" containerName="payload-pre-processing"
Jul 26 18:22:54.495443 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:54.493244    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7348fd2f-eba6-4334-9555-483772fe1a22" containerName="payload-pre-processing"
Jul 26 18:22:54.495443 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:54.493252    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="533a155f-73fd-485d-adc7-a122d5415104" containerName="payload-processing"
Jul 26 18:22:54.495443 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:54.493260    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="d05a74cc-01db-4d99-9e5c-ae48a95597de" containerName="maas-api"
Jul 26 18:22:54.499192 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:54.497498    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9"
Jul 26 18:22:54.500400 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:54.500373    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-614abb29-serving-cert\""
Jul 26 18:22:54.511004 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:54.510972    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9"]
Jul 26 18:22:54.542080 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:54.542048    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flksr\" (UniqueName: \"kubernetes.io/projected/f33bcb81-92a9-434d-b64a-5a0a3153a01f-kube-api-access-flksr\") pod \"maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9\" (UID: \"f33bcb81-92a9-434d-b64a-5a0a3153a01f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9"
Jul 26 18:22:54.542268 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:54.542160    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f33bcb81-92a9-434d-b64a-5a0a3153a01f-openshift-service-ca\") pod \"maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9\" (UID: \"f33bcb81-92a9-434d-b64a-5a0a3153a01f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9"
Jul 26 18:22:54.542268 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:54.542221    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f33bcb81-92a9-434d-b64a-5a0a3153a01f-maas-api-tls\") pod \"maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9\" (UID: \"f33bcb81-92a9-434d-b64a-5a0a3153a01f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9"
Jul 26 18:22:54.643103 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:54.643064    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f33bcb81-92a9-434d-b64a-5a0a3153a01f-openshift-service-ca\") pod \"maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9\" (UID: \"f33bcb81-92a9-434d-b64a-5a0a3153a01f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9"
Jul 26 18:22:54.643335 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:54.643142    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f33bcb81-92a9-434d-b64a-5a0a3153a01f-maas-api-tls\") pod \"maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9\" (UID: \"f33bcb81-92a9-434d-b64a-5a0a3153a01f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9"
Jul 26 18:22:54.643335 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:54.643174    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flksr\" (UniqueName: \"kubernetes.io/projected/f33bcb81-92a9-434d-b64a-5a0a3153a01f-kube-api-access-flksr\") pod \"maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9\" (UID: \"f33bcb81-92a9-434d-b64a-5a0a3153a01f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9"
Jul 26 18:22:54.643335 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:22:54.643251    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-serving-cert: secret "maas-api-e2e-mt-614abb29-serving-cert" not found
Jul 26 18:22:54.643502 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:22:54.643344    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f33bcb81-92a9-434d-b64a-5a0a3153a01f-maas-api-tls podName:f33bcb81-92a9-434d-b64a-5a0a3153a01f nodeName:}" failed. No retries permitted until 2026-07-26 18:22:55.143319738 +0000 UTC m=+2031.795510578 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f33bcb81-92a9-434d-b64a-5a0a3153a01f-maas-api-tls") pod "maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9" (UID: "f33bcb81-92a9-434d-b64a-5a0a3153a01f") : secret "maas-api-e2e-mt-614abb29-serving-cert" not found
Jul 26 18:22:54.643813 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:54.643794    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f33bcb81-92a9-434d-b64a-5a0a3153a01f-openshift-service-ca\") pod \"maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9\" (UID: \"f33bcb81-92a9-434d-b64a-5a0a3153a01f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9"
Jul 26 18:22:54.652276 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:54.652247    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flksr\" (UniqueName: \"kubernetes.io/projected/f33bcb81-92a9-434d-b64a-5a0a3153a01f-kube-api-access-flksr\") pod \"maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9\" (UID: \"f33bcb81-92a9-434d-b64a-5a0a3153a01f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9"
Jul 26 18:22:55.148173 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.148134    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f33bcb81-92a9-434d-b64a-5a0a3153a01f-maas-api-tls\") pod \"maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9\" (UID: \"f33bcb81-92a9-434d-b64a-5a0a3153a01f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9"
Jul 26 18:22:55.150611 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.150581    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f33bcb81-92a9-434d-b64a-5a0a3153a01f-maas-api-tls\") pod \"maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9\" (UID: \"f33bcb81-92a9-434d-b64a-5a0a3153a01f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9"
Jul 26 18:22:55.302186 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.302154    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg"]
Jul 26 18:22:55.305707 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.305691    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg"
Jul 26 18:22:55.310504 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.310482    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-614abb29\""
Jul 26 18:22:55.329697 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.329672    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg"]
Jul 26 18:22:55.349430 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.349403    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h294k\" (UniqueName: \"kubernetes.io/projected/b464508b-5537-40fd-9736-9fa72c9b3385-kube-api-access-h294k\") pod \"payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg\" (UID: \"b464508b-5537-40fd-9736-9fa72c9b3385\") " pod="openshift-ingress/payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg"
Jul 26 18:22:55.349561 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.349530    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b464508b-5537-40fd-9736-9fa72c9b3385-plugins-config-volume\") pod \"payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg\" (UID: \"b464508b-5537-40fd-9736-9fa72c9b3385\") " pod="openshift-ingress/payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg"
Jul 26 18:22:55.389865 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.389836    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt"]
Jul 26 18:22:55.393263 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.393247    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt"
Jul 26 18:22:55.396447 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.396426    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-614abb29-dockercfg-q2n58\""
Jul 26 18:22:55.410478 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.410453    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt"]
Jul 26 18:22:55.410595 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.410557    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9"
Jul 26 18:22:55.449959 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.449923    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b464508b-5537-40fd-9736-9fa72c9b3385-plugins-config-volume\") pod \"payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg\" (UID: \"b464508b-5537-40fd-9736-9fa72c9b3385\") " pod="openshift-ingress/payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg"
Jul 26 18:22:55.449959 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.449964    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pt8x\" (UniqueName: \"kubernetes.io/projected/7e30c2b3-8388-420e-9c67-35f685d223ce-kube-api-access-8pt8x\") pod \"payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt\" (UID: \"7e30c2b3-8388-420e-9c67-35f685d223ce\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt"
Jul 26 18:22:55.450215 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.450014    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h294k\" (UniqueName: \"kubernetes.io/projected/b464508b-5537-40fd-9736-9fa72c9b3385-kube-api-access-h294k\") pod \"payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg\" (UID: \"b464508b-5537-40fd-9736-9fa72c9b3385\") " pod="openshift-ingress/payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg"
Jul 26 18:22:55.450215 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.450056    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e30c2b3-8388-420e-9c67-35f685d223ce-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt\" (UID: \"7e30c2b3-8388-420e-9c67-35f685d223ce\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt"
Jul 26 18:22:55.450951 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.450925    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b464508b-5537-40fd-9736-9fa72c9b3385-plugins-config-volume\") pod \"payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg\" (UID: \"b464508b-5537-40fd-9736-9fa72c9b3385\") " pod="openshift-ingress/payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg"
Jul 26 18:22:55.465743 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.465667    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h294k\" (UniqueName: \"kubernetes.io/projected/b464508b-5537-40fd-9736-9fa72c9b3385-kube-api-access-h294k\") pod \"payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg\" (UID: \"b464508b-5537-40fd-9736-9fa72c9b3385\") " pod="openshift-ingress/payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg"
Jul 26 18:22:55.550594 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.550549    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pt8x\" (UniqueName: \"kubernetes.io/projected/7e30c2b3-8388-420e-9c67-35f685d223ce-kube-api-access-8pt8x\") pod \"payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt\" (UID: \"7e30c2b3-8388-420e-9c67-35f685d223ce\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt"
Jul 26 18:22:55.551070 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.550654    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e30c2b3-8388-420e-9c67-35f685d223ce-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt\" (UID: \"7e30c2b3-8388-420e-9c67-35f685d223ce\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt"
Jul 26 18:22:55.551475 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.551451    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e30c2b3-8388-420e-9c67-35f685d223ce-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt\" (UID: \"7e30c2b3-8388-420e-9c67-35f685d223ce\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt"
Jul 26 18:22:55.560853 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.560827    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pt8x\" (UniqueName: \"kubernetes.io/projected/7e30c2b3-8388-420e-9c67-35f685d223ce-kube-api-access-8pt8x\") pod \"payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt\" (UID: \"7e30c2b3-8388-420e-9c67-35f685d223ce\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt"
Jul 26 18:22:55.564938 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.564917    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9"]
Jul 26 18:22:55.566694 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:22:55.566672    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf33bcb81_92a9_434d_b64a_5a0a3153a01f.slice/crio-0823ad6ae18a2151214118c07c7c0cd5f2771b04898cc3b3b4547dba6f31663c WatchSource:0}: Error finding container 0823ad6ae18a2151214118c07c7c0cd5f2771b04898cc3b3b4547dba6f31663c: Status 404 returned error can't find the container with id 0823ad6ae18a2151214118c07c7c0cd5f2771b04898cc3b3b4547dba6f31663c
Jul 26 18:22:55.567998 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.567983    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 18:22:55.616019 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.615978    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg"
Jul 26 18:22:55.703585 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.703544    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt"
Jul 26 18:22:55.819329 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.819299    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg"]
Jul 26 18:22:55.828645 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:22:55.828614    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb464508b_5537_40fd_9736_9fa72c9b3385.slice/crio-a60509d53ff8083c70b119a560fbc6e9788e44c138a53f0eaf6a24a514913e43 WatchSource:0}: Error finding container a60509d53ff8083c70b119a560fbc6e9788e44c138a53f0eaf6a24a514913e43: Status 404 returned error can't find the container with id a60509d53ff8083c70b119a560fbc6e9788e44c138a53f0eaf6a24a514913e43
Jul 26 18:22:55.857034 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:55.857003    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt"]
Jul 26 18:22:55.857639 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:22:55.857611    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e30c2b3_8388_420e_9c67_35f685d223ce.slice/crio-369e9489afd4aba30768a77d22da0f79a9c5d897327921b209b48524a55f1dd1 WatchSource:0}: Error finding container 369e9489afd4aba30768a77d22da0f79a9c5d897327921b209b48524a55f1dd1: Status 404 returned error can't find the container with id 369e9489afd4aba30768a77d22da0f79a9c5d897327921b209b48524a55f1dd1
Jul 26 18:22:56.521034 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:56.521000    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt" event={"ID":"7e30c2b3-8388-420e-9c67-35f685d223ce","Type":"ContainerStarted","Data":"205f0fe5837630f5c1c561e750b2e72f190d0b0874c30852b7288b8161d03976"}
Jul 26 18:22:56.521248 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:56.521060    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt"
Jul 26 18:22:56.521248 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:56.521078    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt" event={"ID":"7e30c2b3-8388-420e-9c67-35f685d223ce","Type":"ContainerStarted","Data":"369e9489afd4aba30768a77d22da0f79a9c5d897327921b209b48524a55f1dd1"}
Jul 26 18:22:56.522637 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:56.522605    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9" event={"ID":"f33bcb81-92a9-434d-b64a-5a0a3153a01f","Type":"ContainerStarted","Data":"041d3a97374a24911f7bbaee4271afaf9acbd9841fa7d0ba2e2ac129049cd73e"}
Jul 26 18:22:56.522759 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:56.522655    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9" event={"ID":"f33bcb81-92a9-434d-b64a-5a0a3153a01f","Type":"ContainerStarted","Data":"0823ad6ae18a2151214118c07c7c0cd5f2771b04898cc3b3b4547dba6f31663c"}
Jul 26 18:22:56.522759 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:56.522742    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9"
Jul 26 18:22:56.524039 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:56.524018    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg" event={"ID":"b464508b-5537-40fd-9736-9fa72c9b3385","Type":"ContainerStarted","Data":"adca6caf4f737bf29b1222cea0472b446960fffc37258ce419169187bec356eb"}
Jul 26 18:22:56.524135 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:56.524059    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg" event={"ID":"b464508b-5537-40fd-9736-9fa72c9b3385","Type":"ContainerStarted","Data":"a60509d53ff8083c70b119a560fbc6e9788e44c138a53f0eaf6a24a514913e43"}
Jul 26 18:22:56.524135 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:56.524089    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg"
Jul 26 18:22:56.538986 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:56.538935    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt" podStartSLOduration=1.538918003 podStartE2EDuration="1.538918003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:22:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:22:56.537855871 +0000 UTC m=+2033.190046730" watchObservedRunningTime="2026-07-26 18:22:56.538918003 +0000 UTC m=+2033.191108860"
Jul 26 18:22:56.556158 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:56.556078    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9" podStartSLOduration=2.211812606 podStartE2EDuration="2.556063723s" totalImagesPullingTime="344.251117ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:22:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:22:56.554849566 +0000 UTC m=+2033.207040424" watchObservedRunningTime="2026-07-26 18:22:56.556063723 +0000 UTC m=+2033.208254580"
Jul 26 18:22:56.574393 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:22:56.574343    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg" podStartSLOduration=1.57432757 podStartE2EDuration="1.57432757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:22:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:22:56.572627002 +0000 UTC m=+2033.224817863" watchObservedRunningTime="2026-07-26 18:22:56.57432757 +0000 UTC m=+2033.226518427"
Jul 26 18:23:02.537029 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:02.536997    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9"
Jul 26 18:23:07.533927 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:07.533896    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt"
Jul 26 18:23:07.534352 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:07.534058    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg"
Jul 26 18:23:43.439500 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.439409    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9"]
Jul 26 18:23:43.440050 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.439653    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9" podUID="f33bcb81-92a9-434d-b64a-5a0a3153a01f" containerName="maas-api" containerID="cri-o://041d3a97374a24911f7bbaee4271afaf9acbd9841fa7d0ba2e2ac129049cd73e" gracePeriod=30
Jul 26 18:23:43.479787 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.479753    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg"]
Jul 26 18:23:43.480018 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.479972    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg" podUID="b464508b-5537-40fd-9736-9fa72c9b3385" containerName="payload-processing" containerID="cri-o://adca6caf4f737bf29b1222cea0472b446960fffc37258ce419169187bec356eb" gracePeriod=30
Jul 26 18:23:43.485378 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.485343    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt"]
Jul 26 18:23:43.485621 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.485577    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt" podUID="7e30c2b3-8388-420e-9c67-35f685d223ce" containerName="payload-pre-processing" containerID="cri-o://205f0fe5837630f5c1c561e750b2e72f190d0b0874c30852b7288b8161d03976" gracePeriod=30
Jul 26 18:23:43.737754 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.737720    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg"
Jul 26 18:23:43.759700 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.759675    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9"
Jul 26 18:23:43.782637 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.782610    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt"
Jul 26 18:23:43.797056 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.797032    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h294k\" (UniqueName: \"kubernetes.io/projected/b464508b-5537-40fd-9736-9fa72c9b3385-kube-api-access-h294k\") pod \"b464508b-5537-40fd-9736-9fa72c9b3385\" (UID: \"b464508b-5537-40fd-9736-9fa72c9b3385\") "
Jul 26 18:23:43.797251 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.797071    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b464508b-5537-40fd-9736-9fa72c9b3385-plugins-config-volume\") pod \"b464508b-5537-40fd-9736-9fa72c9b3385\" (UID: \"b464508b-5537-40fd-9736-9fa72c9b3385\") "
Jul 26 18:23:43.797562 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.797532    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b464508b-5537-40fd-9736-9fa72c9b3385-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b464508b-5537-40fd-9736-9fa72c9b3385" (UID: "b464508b-5537-40fd-9736-9fa72c9b3385"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:23:43.798310 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.798286    2570 generic.go:356] "Generic (PLEG): container finished" podID="f33bcb81-92a9-434d-b64a-5a0a3153a01f" containerID="041d3a97374a24911f7bbaee4271afaf9acbd9841fa7d0ba2e2ac129049cd73e" exitCode=0
Jul 26 18:23:43.798438 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.798358    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9"
Jul 26 18:23:43.798438 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.798376    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9" event={"ID":"f33bcb81-92a9-434d-b64a-5a0a3153a01f","Type":"ContainerDied","Data":"041d3a97374a24911f7bbaee4271afaf9acbd9841fa7d0ba2e2ac129049cd73e"}
Jul 26 18:23:43.798438 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.798437    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9" event={"ID":"f33bcb81-92a9-434d-b64a-5a0a3153a01f","Type":"ContainerDied","Data":"0823ad6ae18a2151214118c07c7c0cd5f2771b04898cc3b3b4547dba6f31663c"}
Jul 26 18:23:43.798605 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.798461    2570 scope.go:117] "RemoveContainer" containerID="041d3a97374a24911f7bbaee4271afaf9acbd9841fa7d0ba2e2ac129049cd73e"
Jul 26 18:23:43.799601 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.799574    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b464508b-5537-40fd-9736-9fa72c9b3385-kube-api-access-h294k" (OuterVolumeSpecName: "kube-api-access-h294k") pod "b464508b-5537-40fd-9736-9fa72c9b3385" (UID: "b464508b-5537-40fd-9736-9fa72c9b3385"). InnerVolumeSpecName "kube-api-access-h294k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:23:43.800028 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.799999    2570 generic.go:356] "Generic (PLEG): container finished" podID="b464508b-5537-40fd-9736-9fa72c9b3385" containerID="adca6caf4f737bf29b1222cea0472b446960fffc37258ce419169187bec356eb" exitCode=0
Jul 26 18:23:43.800155 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.800031    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg" event={"ID":"b464508b-5537-40fd-9736-9fa72c9b3385","Type":"ContainerDied","Data":"adca6caf4f737bf29b1222cea0472b446960fffc37258ce419169187bec356eb"}
Jul 26 18:23:43.800155 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.800088    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg" event={"ID":"b464508b-5537-40fd-9736-9fa72c9b3385","Type":"ContainerDied","Data":"a60509d53ff8083c70b119a560fbc6e9788e44c138a53f0eaf6a24a514913e43"}
Jul 26 18:23:43.800155 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.800039    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg"
Jul 26 18:23:43.801622 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.801599    2570 generic.go:356] "Generic (PLEG): container finished" podID="7e30c2b3-8388-420e-9c67-35f685d223ce" containerID="205f0fe5837630f5c1c561e750b2e72f190d0b0874c30852b7288b8161d03976" exitCode=0
Jul 26 18:23:43.801736 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.801642    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt"
Jul 26 18:23:43.801736 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.801646    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt" event={"ID":"7e30c2b3-8388-420e-9c67-35f685d223ce","Type":"ContainerDied","Data":"205f0fe5837630f5c1c561e750b2e72f190d0b0874c30852b7288b8161d03976"}
Jul 26 18:23:43.801736 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.801682    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt" event={"ID":"7e30c2b3-8388-420e-9c67-35f685d223ce","Type":"ContainerDied","Data":"369e9489afd4aba30768a77d22da0f79a9c5d897327921b209b48524a55f1dd1"}
Jul 26 18:23:43.809077 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.809060    2570 scope.go:117] "RemoveContainer" containerID="041d3a97374a24911f7bbaee4271afaf9acbd9841fa7d0ba2e2ac129049cd73e"
Jul 26 18:23:43.809435 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:23:43.809404    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"041d3a97374a24911f7bbaee4271afaf9acbd9841fa7d0ba2e2ac129049cd73e\": container with ID starting with 041d3a97374a24911f7bbaee4271afaf9acbd9841fa7d0ba2e2ac129049cd73e not found: ID does not exist" containerID="041d3a97374a24911f7bbaee4271afaf9acbd9841fa7d0ba2e2ac129049cd73e"
Jul 26 18:23:43.809523 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.809452    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"041d3a97374a24911f7bbaee4271afaf9acbd9841fa7d0ba2e2ac129049cd73e"} err="failed to get container status \"041d3a97374a24911f7bbaee4271afaf9acbd9841fa7d0ba2e2ac129049cd73e\": rpc error: code = NotFound desc = could not find container \"041d3a97374a24911f7bbaee4271afaf9acbd9841fa7d0ba2e2ac129049cd73e\": container with ID starting with 041d3a97374a24911f7bbaee4271afaf9acbd9841fa7d0ba2e2ac129049cd73e not found: ID does not exist"
Jul 26 18:23:43.809523 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.809476    2570 scope.go:117] "RemoveContainer" containerID="adca6caf4f737bf29b1222cea0472b446960fffc37258ce419169187bec356eb"
Jul 26 18:23:43.821289 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.821266    2570 scope.go:117] "RemoveContainer" containerID="adca6caf4f737bf29b1222cea0472b446960fffc37258ce419169187bec356eb"
Jul 26 18:23:43.821615 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:23:43.821594    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adca6caf4f737bf29b1222cea0472b446960fffc37258ce419169187bec356eb\": container with ID starting with adca6caf4f737bf29b1222cea0472b446960fffc37258ce419169187bec356eb not found: ID does not exist" containerID="adca6caf4f737bf29b1222cea0472b446960fffc37258ce419169187bec356eb"
Jul 26 18:23:43.821697 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.821620    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adca6caf4f737bf29b1222cea0472b446960fffc37258ce419169187bec356eb"} err="failed to get container status \"adca6caf4f737bf29b1222cea0472b446960fffc37258ce419169187bec356eb\": rpc error: code = NotFound desc = could not find container \"adca6caf4f737bf29b1222cea0472b446960fffc37258ce419169187bec356eb\": container with ID starting with adca6caf4f737bf29b1222cea0472b446960fffc37258ce419169187bec356eb not found: ID does not exist"
Jul 26 18:23:43.821697 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.821638    2570 scope.go:117] "RemoveContainer" containerID="205f0fe5837630f5c1c561e750b2e72f190d0b0874c30852b7288b8161d03976"
Jul 26 18:23:43.831922 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.831903    2570 scope.go:117] "RemoveContainer" containerID="205f0fe5837630f5c1c561e750b2e72f190d0b0874c30852b7288b8161d03976"
Jul 26 18:23:43.832324 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:23:43.832297    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"205f0fe5837630f5c1c561e750b2e72f190d0b0874c30852b7288b8161d03976\": container with ID starting with 205f0fe5837630f5c1c561e750b2e72f190d0b0874c30852b7288b8161d03976 not found: ID does not exist" containerID="205f0fe5837630f5c1c561e750b2e72f190d0b0874c30852b7288b8161d03976"
Jul 26 18:23:43.832389 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.832336    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"205f0fe5837630f5c1c561e750b2e72f190d0b0874c30852b7288b8161d03976"} err="failed to get container status \"205f0fe5837630f5c1c561e750b2e72f190d0b0874c30852b7288b8161d03976\": rpc error: code = NotFound desc = could not find container \"205f0fe5837630f5c1c561e750b2e72f190d0b0874c30852b7288b8161d03976\": container with ID starting with 205f0fe5837630f5c1c561e750b2e72f190d0b0874c30852b7288b8161d03976 not found: ID does not exist"
Jul 26 18:23:43.848948 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.848916    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg"]
Jul 26 18:23:43.853916 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.853893    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-614abb29-688f596ff4-xpkmg"]
Jul 26 18:23:43.898601 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.898569    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flksr\" (UniqueName: \"kubernetes.io/projected/f33bcb81-92a9-434d-b64a-5a0a3153a01f-kube-api-access-flksr\") pod \"f33bcb81-92a9-434d-b64a-5a0a3153a01f\" (UID: \"f33bcb81-92a9-434d-b64a-5a0a3153a01f\") "
Jul 26 18:23:43.898747 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.898608    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f33bcb81-92a9-434d-b64a-5a0a3153a01f-openshift-service-ca\") pod \"f33bcb81-92a9-434d-b64a-5a0a3153a01f\" (UID: \"f33bcb81-92a9-434d-b64a-5a0a3153a01f\") "
Jul 26 18:23:43.898747 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.898686    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pt8x\" (UniqueName: \"kubernetes.io/projected/7e30c2b3-8388-420e-9c67-35f685d223ce-kube-api-access-8pt8x\") pod \"7e30c2b3-8388-420e-9c67-35f685d223ce\" (UID: \"7e30c2b3-8388-420e-9c67-35f685d223ce\") "
Jul 26 18:23:43.898747 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.898719    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f33bcb81-92a9-434d-b64a-5a0a3153a01f-maas-api-tls\") pod \"f33bcb81-92a9-434d-b64a-5a0a3153a01f\" (UID: \"f33bcb81-92a9-434d-b64a-5a0a3153a01f\") "
Jul 26 18:23:43.898866 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.898764    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e30c2b3-8388-420e-9c67-35f685d223ce-plugins-config-volume\") pod \"7e30c2b3-8388-420e-9c67-35f685d223ce\" (UID: \"7e30c2b3-8388-420e-9c67-35f685d223ce\") "
Jul 26 18:23:43.899054 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.899034    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h294k\" (UniqueName: \"kubernetes.io/projected/b464508b-5537-40fd-9736-9fa72c9b3385-kube-api-access-h294k\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:23:43.899136 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.899059    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b464508b-5537-40fd-9736-9fa72c9b3385-plugins-config-volume\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:23:43.899136 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.899092    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f33bcb81-92a9-434d-b64a-5a0a3153a01f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f33bcb81-92a9-434d-b64a-5a0a3153a01f" (UID: "f33bcb81-92a9-434d-b64a-5a0a3153a01f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:23:43.899267 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.899241    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e30c2b3-8388-420e-9c67-35f685d223ce-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7e30c2b3-8388-420e-9c67-35f685d223ce" (UID: "7e30c2b3-8388-420e-9c67-35f685d223ce"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:23:43.900929 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.900906    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f33bcb81-92a9-434d-b64a-5a0a3153a01f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f33bcb81-92a9-434d-b64a-5a0a3153a01f" (UID: "f33bcb81-92a9-434d-b64a-5a0a3153a01f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:23:43.901015 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.900950    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e30c2b3-8388-420e-9c67-35f685d223ce-kube-api-access-8pt8x" (OuterVolumeSpecName: "kube-api-access-8pt8x") pod "7e30c2b3-8388-420e-9c67-35f685d223ce" (UID: "7e30c2b3-8388-420e-9c67-35f685d223ce"). InnerVolumeSpecName "kube-api-access-8pt8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:23:43.901015 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.900984    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f33bcb81-92a9-434d-b64a-5a0a3153a01f-kube-api-access-flksr" (OuterVolumeSpecName: "kube-api-access-flksr") pod "f33bcb81-92a9-434d-b64a-5a0a3153a01f" (UID: "f33bcb81-92a9-434d-b64a-5a0a3153a01f"). InnerVolumeSpecName "kube-api-access-flksr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:23:43.999690 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.999589    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pt8x\" (UniqueName: \"kubernetes.io/projected/7e30c2b3-8388-420e-9c67-35f685d223ce-kube-api-access-8pt8x\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:23:43.999690 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.999625    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f33bcb81-92a9-434d-b64a-5a0a3153a01f-maas-api-tls\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:23:43.999690 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.999638    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e30c2b3-8388-420e-9c67-35f685d223ce-plugins-config-volume\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:23:43.999690 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.999651    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flksr\" (UniqueName: \"kubernetes.io/projected/f33bcb81-92a9-434d-b64a-5a0a3153a01f-kube-api-access-flksr\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:23:43.999690 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:43.999662    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f33bcb81-92a9-434d-b64a-5a0a3153a01f-openshift-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:23:44.122346 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:44.122312    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9"]
Jul 26 18:23:44.126157 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:44.126125    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-614abb29-6cb66b8857-dzkz9"]
Jul 26 18:23:44.136156 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:44.136106    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt"]
Jul 26 18:23:44.138168 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:44.138146    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-614abb29-597f55fdb4-cp6pt"]
Jul 26 18:23:45.838100 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:45.838067    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e30c2b3-8388-420e-9c67-35f685d223ce" path="/var/lib/kubelet/pods/7e30c2b3-8388-420e-9c67-35f685d223ce/volumes"
Jul 26 18:23:45.838496 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:45.838434    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b464508b-5537-40fd-9736-9fa72c9b3385" path="/var/lib/kubelet/pods/b464508b-5537-40fd-9736-9fa72c9b3385/volumes"
Jul 26 18:23:45.838767 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:23:45.838755    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f33bcb81-92a9-434d-b64a-5a0a3153a01f" path="/var/lib/kubelet/pods/f33bcb81-92a9-434d-b64a-5a0a3153a01f/volumes"
Jul 26 18:24:03.989189 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:24:03.989160    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 18:24:03.995403 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:24:03.995378    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 18:24:04.135718 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:24:04.135686    2570 scope.go:117] "RemoveContainer" containerID="6855f7408ad9232a3eda0268308dccd87ace905f7064f56a96424c2ca246c2b4"
Jul 26 18:25:04.169049 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:04.168965    2570 scope.go:117] "RemoveContainer" containerID="b665600242cfa969495e925bd6c7e29eb8dc48aa395463c227261693dc57a3d0"
Jul 26 18:25:43.468018 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:43.467981    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4"]
Jul 26 18:25:43.468508 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:43.468366    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b464508b-5537-40fd-9736-9fa72c9b3385" containerName="payload-processing"
Jul 26 18:25:43.468508 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:43.468381    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b464508b-5537-40fd-9736-9fa72c9b3385" containerName="payload-processing"
Jul 26 18:25:43.468508 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:43.468388    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e30c2b3-8388-420e-9c67-35f685d223ce" containerName="payload-pre-processing"
Jul 26 18:25:43.468508 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:43.468393    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e30c2b3-8388-420e-9c67-35f685d223ce" containerName="payload-pre-processing"
Jul 26 18:25:43.468508 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:43.468401    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f33bcb81-92a9-434d-b64a-5a0a3153a01f" containerName="maas-api"
Jul 26 18:25:43.468508 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:43.468407    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f33bcb81-92a9-434d-b64a-5a0a3153a01f" containerName="maas-api"
Jul 26 18:25:43.468508 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:43.468464    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b464508b-5537-40fd-9736-9fa72c9b3385" containerName="payload-processing"
Jul 26 18:25:43.468508 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:43.468475    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f33bcb81-92a9-434d-b64a-5a0a3153a01f" containerName="maas-api"
Jul 26 18:25:43.468508 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:43.468480    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e30c2b3-8388-420e-9c67-35f685d223ce" containerName="payload-pre-processing"
Jul 26 18:25:43.471557 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:43.471535    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4"
Jul 26 18:25:43.473586 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:43.473568    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-bb8f83-serving-cert\""
Jul 26 18:25:43.483371 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:43.483344    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4"]
Jul 26 18:25:43.542501 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:43.542464    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz5vz\" (UniqueName: \"kubernetes.io/projected/52a8494c-909b-4a5a-85fe-794e61d46fa9-kube-api-access-fz5vz\") pod \"maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4\" (UID: \"52a8494c-909b-4a5a-85fe-794e61d46fa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4"
Jul 26 18:25:43.542682 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:43.542509    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52a8494c-909b-4a5a-85fe-794e61d46fa9-maas-api-tls\") pod \"maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4\" (UID: \"52a8494c-909b-4a5a-85fe-794e61d46fa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4"
Jul 26 18:25:43.542682 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:43.542618    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52a8494c-909b-4a5a-85fe-794e61d46fa9-openshift-service-ca\") pod \"maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4\" (UID: \"52a8494c-909b-4a5a-85fe-794e61d46fa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4"
Jul 26 18:25:43.643816 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:43.643779    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52a8494c-909b-4a5a-85fe-794e61d46fa9-openshift-service-ca\") pod \"maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4\" (UID: \"52a8494c-909b-4a5a-85fe-794e61d46fa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4"
Jul 26 18:25:43.644036 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:43.643839    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz5vz\" (UniqueName: \"kubernetes.io/projected/52a8494c-909b-4a5a-85fe-794e61d46fa9-kube-api-access-fz5vz\") pod \"maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4\" (UID: \"52a8494c-909b-4a5a-85fe-794e61d46fa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4"
Jul 26 18:25:43.644036 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:43.643862    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52a8494c-909b-4a5a-85fe-794e61d46fa9-maas-api-tls\") pod \"maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4\" (UID: \"52a8494c-909b-4a5a-85fe-794e61d46fa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4"
Jul 26 18:25:43.644036 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:25:43.643978    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-serving-cert: secret "maas-api-e2e-inf-a-bb8f83-serving-cert" not found
Jul 26 18:25:43.644282 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:25:43.644043    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52a8494c-909b-4a5a-85fe-794e61d46fa9-maas-api-tls podName:52a8494c-909b-4a5a-85fe-794e61d46fa9 nodeName:}" failed. No retries permitted until 2026-07-26 18:25:44.144026572 +0000 UTC m=+2200.796217409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/52a8494c-909b-4a5a-85fe-794e61d46fa9-maas-api-tls") pod "maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4" (UID: "52a8494c-909b-4a5a-85fe-794e61d46fa9") : secret "maas-api-e2e-inf-a-bb8f83-serving-cert" not found
Jul 26 18:25:43.644596 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:43.644570    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52a8494c-909b-4a5a-85fe-794e61d46fa9-openshift-service-ca\") pod \"maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4\" (UID: \"52a8494c-909b-4a5a-85fe-794e61d46fa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4"
Jul 26 18:25:43.653523 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:43.653497    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz5vz\" (UniqueName: \"kubernetes.io/projected/52a8494c-909b-4a5a-85fe-794e61d46fa9-kube-api-access-fz5vz\") pod \"maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4\" (UID: \"52a8494c-909b-4a5a-85fe-794e61d46fa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4"
Jul 26 18:25:44.148290 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:44.148255    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52a8494c-909b-4a5a-85fe-794e61d46fa9-maas-api-tls\") pod \"maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4\" (UID: \"52a8494c-909b-4a5a-85fe-794e61d46fa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4"
Jul 26 18:25:44.150698 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:44.150676    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52a8494c-909b-4a5a-85fe-794e61d46fa9-maas-api-tls\") pod \"maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4\" (UID: \"52a8494c-909b-4a5a-85fe-794e61d46fa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4"
Jul 26 18:25:44.221307 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:44.221271    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w"]
Jul 26 18:25:44.224867 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:44.224850    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w"
Jul 26 18:25:44.226959 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:44.226938    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-bb8f83\""
Jul 26 18:25:44.227090 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:44.227071    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-bb8f83-dockercfg-f4tgs\""
Jul 26 18:25:44.235023 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:44.234995    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w"]
Jul 26 18:25:44.350522 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:44.350478    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a266d3e-f07a-49f7-b84f-8345273ab92d-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w\" (UID: \"4a266d3e-f07a-49f7-b84f-8345273ab92d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w"
Jul 26 18:25:44.350522 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:44.350527    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnrj9\" (UniqueName: \"kubernetes.io/projected/4a266d3e-f07a-49f7-b84f-8345273ab92d-kube-api-access-cnrj9\") pod \"payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w\" (UID: \"4a266d3e-f07a-49f7-b84f-8345273ab92d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w"
Jul 26 18:25:44.384562 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:44.384532    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4"
Jul 26 18:25:44.452328 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:44.451215    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a266d3e-f07a-49f7-b84f-8345273ab92d-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w\" (UID: \"4a266d3e-f07a-49f7-b84f-8345273ab92d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w"
Jul 26 18:25:44.452328 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:44.451265    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnrj9\" (UniqueName: \"kubernetes.io/projected/4a266d3e-f07a-49f7-b84f-8345273ab92d-kube-api-access-cnrj9\") pod \"payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w\" (UID: \"4a266d3e-f07a-49f7-b84f-8345273ab92d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w"
Jul 26 18:25:44.452328 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:44.452157    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a266d3e-f07a-49f7-b84f-8345273ab92d-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w\" (UID: \"4a266d3e-f07a-49f7-b84f-8345273ab92d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w"
Jul 26 18:25:44.463642 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:44.463613    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnrj9\" (UniqueName: \"kubernetes.io/projected/4a266d3e-f07a-49f7-b84f-8345273ab92d-kube-api-access-cnrj9\") pod \"payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w\" (UID: \"4a266d3e-f07a-49f7-b84f-8345273ab92d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w"
Jul 26 18:25:44.536195 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:44.536150    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w"
Jul 26 18:25:44.667155 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:44.667127    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w"]
Jul 26 18:25:44.668717 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:25:44.668685    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a266d3e_f07a_49f7_b84f_8345273ab92d.slice/crio-f2edd50f2b0fd68a37bd9ff89e4dcbfd5d6e83c371da8412553afefc339f6c44 WatchSource:0}: Error finding container f2edd50f2b0fd68a37bd9ff89e4dcbfd5d6e83c371da8412553afefc339f6c44: Status 404 returned error can't find the container with id f2edd50f2b0fd68a37bd9ff89e4dcbfd5d6e83c371da8412553afefc339f6c44
Jul 26 18:25:44.730735 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:44.730706    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4"]
Jul 26 18:25:44.732046 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:25:44.732015    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52a8494c_909b_4a5a_85fe_794e61d46fa9.slice/crio-7275291557f91c80684cb6c284fc27cdb0c09c7700096fcdfdd51f502fe34174 WatchSource:0}: Error finding container 7275291557f91c80684cb6c284fc27cdb0c09c7700096fcdfdd51f502fe34174: Status 404 returned error can't find the container with id 7275291557f91c80684cb6c284fc27cdb0c09c7700096fcdfdd51f502fe34174
Jul 26 18:25:45.463950 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:45.463836    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w" event={"ID":"4a266d3e-f07a-49f7-b84f-8345273ab92d","Type":"ContainerStarted","Data":"3355d26741851f2e0c00ef87aaa70609541f12239caf914724192876ae475444"}
Jul 26 18:25:45.464146 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:45.463977    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w" event={"ID":"4a266d3e-f07a-49f7-b84f-8345273ab92d","Type":"ContainerStarted","Data":"f2edd50f2b0fd68a37bd9ff89e4dcbfd5d6e83c371da8412553afefc339f6c44"}
Jul 26 18:25:45.464146 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:45.464015    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w"
Jul 26 18:25:45.465423 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:45.465395    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4" event={"ID":"52a8494c-909b-4a5a-85fe-794e61d46fa9","Type":"ContainerStarted","Data":"97c0a084d4805798de92cd624b5e19ad7074a78628f0ee8a4466d60151148448"}
Jul 26 18:25:45.465528 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:45.465447    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4" event={"ID":"52a8494c-909b-4a5a-85fe-794e61d46fa9","Type":"ContainerStarted","Data":"7275291557f91c80684cb6c284fc27cdb0c09c7700096fcdfdd51f502fe34174"}
Jul 26 18:25:45.465528 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:45.465476    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4"
Jul 26 18:25:45.481245 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:45.481194    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w" podStartSLOduration=1.481177647 podStartE2EDuration="1.481177647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:25:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:25:45.480016001 +0000 UTC m=+2202.132206886" watchObservedRunningTime="2026-07-26 18:25:45.481177647 +0000 UTC m=+2202.133368505"
Jul 26 18:25:45.496898 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:45.496845    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4" podStartSLOduration=2.10207293 podStartE2EDuration="2.496828104s" totalImagesPullingTime="394.755174ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:25:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:25:45.495805732 +0000 UTC m=+2202.147996593" watchObservedRunningTime="2026-07-26 18:25:45.496828104 +0000 UTC m=+2202.149018962"
Jul 26 18:25:51.390220 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:51.390183    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2"]
Jul 26 18:25:51.394013 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:51.393991    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2"
Jul 26 18:25:51.396270 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:51.396251    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-40c67c-serving-cert\""
Jul 26 18:25:51.406127 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:51.406096    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2"]
Jul 26 18:25:51.476744 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:51.476715    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4"
Jul 26 18:25:51.510715 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:51.510680    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/209496dd-c721-4ea2-adc6-a8a3e58e831d-openshift-service-ca\") pod \"maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2\" (UID: \"209496dd-c721-4ea2-adc6-a8a3e58e831d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2"
Jul 26 18:25:51.510715 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:51.510721    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/209496dd-c721-4ea2-adc6-a8a3e58e831d-maas-api-tls\") pod \"maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2\" (UID: \"209496dd-c721-4ea2-adc6-a8a3e58e831d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2"
Jul 26 18:25:51.510943 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:51.510779    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz4k9\" (UniqueName: \"kubernetes.io/projected/209496dd-c721-4ea2-adc6-a8a3e58e831d-kube-api-access-lz4k9\") pod \"maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2\" (UID: \"209496dd-c721-4ea2-adc6-a8a3e58e831d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2"
Jul 26 18:25:51.611971 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:51.611935    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/209496dd-c721-4ea2-adc6-a8a3e58e831d-openshift-service-ca\") pod \"maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2\" (UID: \"209496dd-c721-4ea2-adc6-a8a3e58e831d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2"
Jul 26 18:25:51.611971 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:51.611971    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/209496dd-c721-4ea2-adc6-a8a3e58e831d-maas-api-tls\") pod \"maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2\" (UID: \"209496dd-c721-4ea2-adc6-a8a3e58e831d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2"
Jul 26 18:25:51.612243 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:51.612030    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz4k9\" (UniqueName: \"kubernetes.io/projected/209496dd-c721-4ea2-adc6-a8a3e58e831d-kube-api-access-lz4k9\") pod \"maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2\" (UID: \"209496dd-c721-4ea2-adc6-a8a3e58e831d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2"
Jul 26 18:25:51.612752 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:51.612725    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/209496dd-c721-4ea2-adc6-a8a3e58e831d-openshift-service-ca\") pod \"maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2\" (UID: \"209496dd-c721-4ea2-adc6-a8a3e58e831d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2"
Jul 26 18:25:51.614461 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:51.614441    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/209496dd-c721-4ea2-adc6-a8a3e58e831d-maas-api-tls\") pod \"maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2\" (UID: \"209496dd-c721-4ea2-adc6-a8a3e58e831d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2"
Jul 26 18:25:51.622807 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:51.622784    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz4k9\" (UniqueName: \"kubernetes.io/projected/209496dd-c721-4ea2-adc6-a8a3e58e831d-kube-api-access-lz4k9\") pod \"maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2\" (UID: \"209496dd-c721-4ea2-adc6-a8a3e58e831d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2"
Jul 26 18:25:51.704892 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:51.704806    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2"
Jul 26 18:25:51.839225 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:51.839192    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2"]
Jul 26 18:25:51.840014 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:25:51.839984    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod209496dd_c721_4ea2_adc6_a8a3e58e831d.slice/crio-5b045bebd995e2881e86cc28aabe6584e8901b7ea781c8aa3c0afacc4504595d WatchSource:0}: Error finding container 5b045bebd995e2881e86cc28aabe6584e8901b7ea781c8aa3c0afacc4504595d: Status 404 returned error can't find the container with id 5b045bebd995e2881e86cc28aabe6584e8901b7ea781c8aa3c0afacc4504595d
Jul 26 18:25:52.088879 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:52.088845    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68"]
Jul 26 18:25:52.093610 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:52.093595    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68"
Jul 26 18:25:52.096780 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:52.096756    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-40c67c\""
Jul 26 18:25:52.096914 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:52.096799    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-40c67c-dockercfg-x7dhg\""
Jul 26 18:25:52.101509 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:52.101485    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68"]
Jul 26 18:25:52.218053 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:52.218007    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdm2l\" (UniqueName: \"kubernetes.io/projected/2aced937-5fc2-4333-8358-339e40a8862a-kube-api-access-zdm2l\") pod \"payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68\" (UID: \"2aced937-5fc2-4333-8358-339e40a8862a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68"
Jul 26 18:25:52.218306 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:52.218140    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2aced937-5fc2-4333-8358-339e40a8862a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68\" (UID: \"2aced937-5fc2-4333-8358-339e40a8862a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68"
Jul 26 18:25:52.319622 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:52.319584    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2aced937-5fc2-4333-8358-339e40a8862a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68\" (UID: \"2aced937-5fc2-4333-8358-339e40a8862a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68"
Jul 26 18:25:52.319798 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:52.319653    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdm2l\" (UniqueName: \"kubernetes.io/projected/2aced937-5fc2-4333-8358-339e40a8862a-kube-api-access-zdm2l\") pod \"payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68\" (UID: \"2aced937-5fc2-4333-8358-339e40a8862a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68"
Jul 26 18:25:52.320341 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:52.320321    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2aced937-5fc2-4333-8358-339e40a8862a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68\" (UID: \"2aced937-5fc2-4333-8358-339e40a8862a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68"
Jul 26 18:25:52.327645 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:52.327622    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdm2l\" (UniqueName: \"kubernetes.io/projected/2aced937-5fc2-4333-8358-339e40a8862a-kube-api-access-zdm2l\") pod \"payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68\" (UID: \"2aced937-5fc2-4333-8358-339e40a8862a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68"
Jul 26 18:25:52.406434 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:52.406352    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68"
Jul 26 18:25:52.506904 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:52.506866    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2" event={"ID":"209496dd-c721-4ea2-adc6-a8a3e58e831d","Type":"ContainerStarted","Data":"94a8c8848a804e15eb59f08f127cac0113bb9efe78d1bbf9458d85ae18c68a2c"}
Jul 26 18:25:52.507101 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:52.506931    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2" event={"ID":"209496dd-c721-4ea2-adc6-a8a3e58e831d","Type":"ContainerStarted","Data":"5b045bebd995e2881e86cc28aabe6584e8901b7ea781c8aa3c0afacc4504595d"}
Jul 26 18:25:52.507101 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:52.507063    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2"
Jul 26 18:25:52.525776 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:52.525728    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2" podStartSLOduration=1.2286818369999999 podStartE2EDuration="1.525712143s" totalImagesPullingTime="297.030306ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:25:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:25:52.523281757 +0000 UTC m=+2209.175472615" watchObservedRunningTime="2026-07-26 18:25:52.525712143 +0000 UTC m=+2209.177903000"
Jul 26 18:25:52.749459 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:52.749428    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68"]
Jul 26 18:25:52.750933 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:25:52.750896    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2aced937_5fc2_4333_8358_339e40a8862a.slice/crio-6fa9bb3587238e11f2969422f0db3173bb5d10c22543e498ddc0c6977c458ab6 WatchSource:0}: Error finding container 6fa9bb3587238e11f2969422f0db3173bb5d10c22543e498ddc0c6977c458ab6: Status 404 returned error can't find the container with id 6fa9bb3587238e11f2969422f0db3173bb5d10c22543e498ddc0c6977c458ab6
Jul 26 18:25:53.514760 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:53.514725    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68" event={"ID":"2aced937-5fc2-4333-8358-339e40a8862a","Type":"ContainerStarted","Data":"06b96bcf3fa290fcc5a6431dc7db1a68af79ddd4116f0ebf640cf5e5e921dafc"}
Jul 26 18:25:53.515233 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:53.514790    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68" event={"ID":"2aced937-5fc2-4333-8358-339e40a8862a","Type":"ContainerStarted","Data":"6fa9bb3587238e11f2969422f0db3173bb5d10c22543e498ddc0c6977c458ab6"}
Jul 26 18:25:53.515233 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:53.514819    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68"
Jul 26 18:25:53.530183 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:53.530134    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68" podStartSLOduration=1.530103604 podStartE2EDuration="1.530103604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:25:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:25:53.528306775 +0000 UTC m=+2210.180497635" watchObservedRunningTime="2026-07-26 18:25:53.530103604 +0000 UTC m=+2210.182294462"
Jul 26 18:25:56.473604 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:56.473576    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w"
Jul 26 18:25:58.520162 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:25:58.520101    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2"
Jul 26 18:26:04.190039 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:04.190000    2570 scope.go:117] "RemoveContainer" containerID="4ced9639cd1e5ee5c3ff6bebab08726621c71c0260d9a649795a0c37bfd5553f"
Jul 26 18:26:04.198609 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:04.198586    2570 scope.go:117] "RemoveContainer" containerID="bc1dfbea6e92d9cbd4d5882b8c800fbd726b5ebcbd12f97c8cb279275cd83f68"
Jul 26 18:26:04.524308 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:04.524281    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68"
Jul 26 18:26:44.102134 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.102032    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"]
Jul 26 18:26:44.105407 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.105390    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:26:44.107582 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.107560    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-40c67c\"/\"default-dockercfg-wnwgd\""
Jul 26 18:26:44.107730 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.107613    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-40c67c\"/\"openshift-service-ca.crt\""
Jul 26 18:26:44.108189 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.108171    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-40c67c\"/\"kube-root-ca.crt\""
Jul 26 18:26:44.108280 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.108206    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-40c67c\"/\"test-model-40c67c-kserve-self-signed-certs\""
Jul 26 18:26:44.113801 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.113780    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"]
Jul 26 18:26:44.255302 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.255262    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-dshm\") pod \"test-model-40c67c-kserve-568f8b775-wbvhf\" (UID: \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\") " pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:26:44.255483 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.255314    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-home\") pod \"test-model-40c67c-kserve-568f8b775-wbvhf\" (UID: \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\") " pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:26:44.255483 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.255346    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-model-cache\") pod \"test-model-40c67c-kserve-568f8b775-wbvhf\" (UID: \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\") " pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:26:44.255483 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.255415    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-tls-certs\") pod \"test-model-40c67c-kserve-568f8b775-wbvhf\" (UID: \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\") " pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:26:44.255483 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.255441    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbvvk\" (UniqueName: \"kubernetes.io/projected/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-kube-api-access-kbvvk\") pod \"test-model-40c67c-kserve-568f8b775-wbvhf\" (UID: \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\") " pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:26:44.255483 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.255467    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-tmp-dir\") pod \"test-model-40c67c-kserve-568f8b775-wbvhf\" (UID: \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\") " pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:26:44.356984 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.356893    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-home\") pod \"test-model-40c67c-kserve-568f8b775-wbvhf\" (UID: \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\") " pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:26:44.356984 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.356938    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-model-cache\") pod \"test-model-40c67c-kserve-568f8b775-wbvhf\" (UID: \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\") " pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:26:44.357204 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.357148    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-tls-certs\") pod \"test-model-40c67c-kserve-568f8b775-wbvhf\" (UID: \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\") " pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:26:44.357204 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.357200    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbvvk\" (UniqueName: \"kubernetes.io/projected/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-kube-api-access-kbvvk\") pod \"test-model-40c67c-kserve-568f8b775-wbvhf\" (UID: \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\") " pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:26:44.357310 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.357227    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-tmp-dir\") pod \"test-model-40c67c-kserve-568f8b775-wbvhf\" (UID: \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\") " pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:26:44.357373 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.357345    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-dshm\") pod \"test-model-40c67c-kserve-568f8b775-wbvhf\" (UID: \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\") " pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:26:44.357425 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.357390    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-home\") pod \"test-model-40c67c-kserve-568f8b775-wbvhf\" (UID: \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\") " pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:26:44.357425 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.357360    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-model-cache\") pod \"test-model-40c67c-kserve-568f8b775-wbvhf\" (UID: \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\") " pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:26:44.357575 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.357554    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-tmp-dir\") pod \"test-model-40c67c-kserve-568f8b775-wbvhf\" (UID: \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\") " pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:26:44.359507 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.359475    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-dshm\") pod \"test-model-40c67c-kserve-568f8b775-wbvhf\" (UID: \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\") " pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:26:44.359707 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.359692    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-tls-certs\") pod \"test-model-40c67c-kserve-568f8b775-wbvhf\" (UID: \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\") " pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:26:44.366318 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.366297    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbvvk\" (UniqueName: \"kubernetes.io/projected/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-kube-api-access-kbvvk\") pod \"test-model-40c67c-kserve-568f8b775-wbvhf\" (UID: \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\") " pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:26:44.416554 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.416511    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:26:44.554476 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.554450    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"]
Jul 26 18:26:44.556083 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:26:44.556056    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c748f9e_70e0_42ba_a00f_aa9b38dcddd2.slice/crio-0b683b653e34eaf22bb0e4b16a9330ee5b33a3b85b250b9297f7adf20d499587 WatchSource:0}: Error finding container 0b683b653e34eaf22bb0e4b16a9330ee5b33a3b85b250b9297f7adf20d499587: Status 404 returned error can't find the container with id 0b683b653e34eaf22bb0e4b16a9330ee5b33a3b85b250b9297f7adf20d499587
Jul 26 18:26:44.803221 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.803185    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf" event={"ID":"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2","Type":"ContainerStarted","Data":"f66a9731e5bd99bd864e25936692a1448d604c3744df59df53d1a62e62371212"}
Jul 26 18:26:44.803395 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.803229    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf" event={"ID":"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2","Type":"ContainerStarted","Data":"0b683b653e34eaf22bb0e4b16a9330ee5b33a3b85b250b9297f7adf20d499587"}
Jul 26 18:26:44.820279 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:44.820231    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf" podStartSLOduration=0.820214619 podStartE2EDuration="820.214619ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:26:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:26:44.819341538 +0000 UTC m=+2261.471532396" watchObservedRunningTime="2026-07-26 18:26:44.820214619 +0000 UTC m=+2261.472405657"
Jul 26 18:26:54.416788 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:54.416757    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:26:54.417327 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:54.416993    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:26:54.429412 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:54.429384    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:26:54.868167 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:26:54.868106    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:27:26.398778 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:26.398737    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"]
Jul 26 18:27:36.947149 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:36.947091    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4"]
Jul 26 18:27:36.947662 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:36.947509    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4" podUID="52a8494c-909b-4a5a-85fe-794e61d46fa9" containerName="maas-api" containerID="cri-o://97c0a084d4805798de92cd624b5e19ad7074a78628f0ee8a4466d60151148448" gracePeriod=30
Jul 26 18:27:37.005281 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:37.005246    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w"]
Jul 26 18:27:37.005575 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:37.005545    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w" podUID="4a266d3e-f07a-49f7-b84f-8345273ab92d" containerName="payload-processing" containerID="cri-o://3355d26741851f2e0c00ef87aaa70609541f12239caf914724192876ae475444" gracePeriod=30
Jul 26 18:27:37.093173 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:37.093140    2570 generic.go:356] "Generic (PLEG): container finished" podID="52a8494c-909b-4a5a-85fe-794e61d46fa9" containerID="97c0a084d4805798de92cd624b5e19ad7074a78628f0ee8a4466d60151148448" exitCode=0
Jul 26 18:27:37.093381 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:37.093226    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4" event={"ID":"52a8494c-909b-4a5a-85fe-794e61d46fa9","Type":"ContainerDied","Data":"97c0a084d4805798de92cd624b5e19ad7074a78628f0ee8a4466d60151148448"}
Jul 26 18:27:37.256745 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:37.256719    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4"
Jul 26 18:27:37.289543 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:37.289514    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w"
Jul 26 18:27:37.333132 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:37.333085    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52a8494c-909b-4a5a-85fe-794e61d46fa9-openshift-service-ca\") pod \"52a8494c-909b-4a5a-85fe-794e61d46fa9\" (UID: \"52a8494c-909b-4a5a-85fe-794e61d46fa9\") "
Jul 26 18:27:37.333330 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:37.333169    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fz5vz\" (UniqueName: \"kubernetes.io/projected/52a8494c-909b-4a5a-85fe-794e61d46fa9-kube-api-access-fz5vz\") pod \"52a8494c-909b-4a5a-85fe-794e61d46fa9\" (UID: \"52a8494c-909b-4a5a-85fe-794e61d46fa9\") "
Jul 26 18:27:37.333330 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:37.333204    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52a8494c-909b-4a5a-85fe-794e61d46fa9-maas-api-tls\") pod \"52a8494c-909b-4a5a-85fe-794e61d46fa9\" (UID: \"52a8494c-909b-4a5a-85fe-794e61d46fa9\") "
Jul 26 18:27:37.333522 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:37.333492    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52a8494c-909b-4a5a-85fe-794e61d46fa9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "52a8494c-909b-4a5a-85fe-794e61d46fa9" (UID: "52a8494c-909b-4a5a-85fe-794e61d46fa9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:27:37.335413 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:37.335383    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52a8494c-909b-4a5a-85fe-794e61d46fa9-kube-api-access-fz5vz" (OuterVolumeSpecName: "kube-api-access-fz5vz") pod "52a8494c-909b-4a5a-85fe-794e61d46fa9" (UID: "52a8494c-909b-4a5a-85fe-794e61d46fa9"). InnerVolumeSpecName "kube-api-access-fz5vz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:27:37.335512 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:37.335408    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52a8494c-909b-4a5a-85fe-794e61d46fa9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "52a8494c-909b-4a5a-85fe-794e61d46fa9" (UID: "52a8494c-909b-4a5a-85fe-794e61d46fa9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:27:37.434207 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:37.434166    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a266d3e-f07a-49f7-b84f-8345273ab92d-plugins-config-volume\") pod \"4a266d3e-f07a-49f7-b84f-8345273ab92d\" (UID: \"4a266d3e-f07a-49f7-b84f-8345273ab92d\") "
Jul 26 18:27:37.434363 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:37.434241    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnrj9\" (UniqueName: \"kubernetes.io/projected/4a266d3e-f07a-49f7-b84f-8345273ab92d-kube-api-access-cnrj9\") pod \"4a266d3e-f07a-49f7-b84f-8345273ab92d\" (UID: \"4a266d3e-f07a-49f7-b84f-8345273ab92d\") "
Jul 26 18:27:37.434528 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:37.434510    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52a8494c-909b-4a5a-85fe-794e61d46fa9-maas-api-tls\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:27:37.434570 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:37.434535    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52a8494c-909b-4a5a-85fe-794e61d46fa9-openshift-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:27:37.434570 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:37.434551    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fz5vz\" (UniqueName: \"kubernetes.io/projected/52a8494c-909b-4a5a-85fe-794e61d46fa9-kube-api-access-fz5vz\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:27:37.434642 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:37.434608    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a266d3e-f07a-49f7-b84f-8345273ab92d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4a266d3e-f07a-49f7-b84f-8345273ab92d" (UID: "4a266d3e-f07a-49f7-b84f-8345273ab92d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:27:37.436352 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:37.436335    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a266d3e-f07a-49f7-b84f-8345273ab92d-kube-api-access-cnrj9" (OuterVolumeSpecName: "kube-api-access-cnrj9") pod "4a266d3e-f07a-49f7-b84f-8345273ab92d" (UID: "4a266d3e-f07a-49f7-b84f-8345273ab92d"). InnerVolumeSpecName "kube-api-access-cnrj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:27:37.535183 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:37.535147    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a266d3e-f07a-49f7-b84f-8345273ab92d-plugins-config-volume\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:27:37.535183 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:37.535177    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnrj9\" (UniqueName: \"kubernetes.io/projected/4a266d3e-f07a-49f7-b84f-8345273ab92d-kube-api-access-cnrj9\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:27:38.100915 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:38.100833    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4"
Jul 26 18:27:38.100915 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:38.100870    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4" event={"ID":"52a8494c-909b-4a5a-85fe-794e61d46fa9","Type":"ContainerDied","Data":"7275291557f91c80684cb6c284fc27cdb0c09c7700096fcdfdd51f502fe34174"}
Jul 26 18:27:38.101437 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:38.100937    2570 scope.go:117] "RemoveContainer" containerID="97c0a084d4805798de92cd624b5e19ad7074a78628f0ee8a4466d60151148448"
Jul 26 18:27:38.102322 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:38.102271    2570 generic.go:356] "Generic (PLEG): container finished" podID="4a266d3e-f07a-49f7-b84f-8345273ab92d" containerID="3355d26741851f2e0c00ef87aaa70609541f12239caf914724192876ae475444" exitCode=0
Jul 26 18:27:38.102444 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:38.102327    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w"
Jul 26 18:27:38.102444 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:38.102335    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w" event={"ID":"4a266d3e-f07a-49f7-b84f-8345273ab92d","Type":"ContainerDied","Data":"3355d26741851f2e0c00ef87aaa70609541f12239caf914724192876ae475444"}
Jul 26 18:27:38.102444 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:38.102370    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w" event={"ID":"4a266d3e-f07a-49f7-b84f-8345273ab92d","Type":"ContainerDied","Data":"f2edd50f2b0fd68a37bd9ff89e4dcbfd5d6e83c371da8412553afefc339f6c44"}
Jul 26 18:27:38.111893 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:38.111866    2570 scope.go:117] "RemoveContainer" containerID="3355d26741851f2e0c00ef87aaa70609541f12239caf914724192876ae475444"
Jul 26 18:27:38.120078 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:38.120000    2570 scope.go:117] "RemoveContainer" containerID="3355d26741851f2e0c00ef87aaa70609541f12239caf914724192876ae475444"
Jul 26 18:27:38.120305 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:27:38.120284    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3355d26741851f2e0c00ef87aaa70609541f12239caf914724192876ae475444\": container with ID starting with 3355d26741851f2e0c00ef87aaa70609541f12239caf914724192876ae475444 not found: ID does not exist" containerID="3355d26741851f2e0c00ef87aaa70609541f12239caf914724192876ae475444"
Jul 26 18:27:38.120357 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:38.120320    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3355d26741851f2e0c00ef87aaa70609541f12239caf914724192876ae475444"} err="failed to get container status \"3355d26741851f2e0c00ef87aaa70609541f12239caf914724192876ae475444\": rpc error: code = NotFound desc = could not find container \"3355d26741851f2e0c00ef87aaa70609541f12239caf914724192876ae475444\": container with ID starting with 3355d26741851f2e0c00ef87aaa70609541f12239caf914724192876ae475444 not found: ID does not exist"
Jul 26 18:27:38.122532 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:38.122510    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4"]
Jul 26 18:27:38.126393 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:38.126370    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bb8f83-5f8586f957-4lsh4"]
Jul 26 18:27:38.135933 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:38.135902    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w"]
Jul 26 18:27:38.138704 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:38.138685    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-bb8f83-bbfdb8dd4-pnl9w"]
Jul 26 18:27:39.838910 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:39.838878    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a266d3e-f07a-49f7-b84f-8345273ab92d" path="/var/lib/kubelet/pods/4a266d3e-f07a-49f7-b84f-8345273ab92d/volumes"
Jul 26 18:27:39.839304 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:39.839282    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52a8494c-909b-4a5a-85fe-794e61d46fa9" path="/var/lib/kubelet/pods/52a8494c-909b-4a5a-85fe-794e61d46fa9/volumes"
Jul 26 18:27:41.418403 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:41.418338    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf" podUID="7c748f9e-70e0-42ba-a00f-aa9b38dcddd2" containerName="main" containerID="cri-o://f66a9731e5bd99bd864e25936692a1448d604c3744df59df53d1a62e62371212" gracePeriod=45
Jul 26 18:27:41.678926 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:41.678863    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:27:41.777702 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:41.777660    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-home\") pod \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\" (UID: \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\") "
Jul 26 18:27:41.777912 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:41.777719    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-tmp-dir\") pod \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\" (UID: \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\") "
Jul 26 18:27:41.777912 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:41.777762    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbvvk\" (UniqueName: \"kubernetes.io/projected/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-kube-api-access-kbvvk\") pod \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\" (UID: \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\") "
Jul 26 18:27:41.777912 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:41.777812    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-dshm\") pod \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\" (UID: \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\") "
Jul 26 18:27:41.777912 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:41.777855    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-model-cache\") pod \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\" (UID: \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\") "
Jul 26 18:27:41.778246 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:41.777983    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-home" (OuterVolumeSpecName: "home") pod "7c748f9e-70e0-42ba-a00f-aa9b38dcddd2" (UID: "7c748f9e-70e0-42ba-a00f-aa9b38dcddd2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:27:41.778246 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:41.778020    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-tls-certs\") pod \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\" (UID: \"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2\") "
Jul 26 18:27:41.778246 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:41.778030    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7c748f9e-70e0-42ba-a00f-aa9b38dcddd2" (UID: "7c748f9e-70e0-42ba-a00f-aa9b38dcddd2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:27:41.778246 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:41.778158    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-model-cache" (OuterVolumeSpecName: "model-cache") pod "7c748f9e-70e0-42ba-a00f-aa9b38dcddd2" (UID: "7c748f9e-70e0-42ba-a00f-aa9b38dcddd2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:27:41.778426 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:41.778378    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-home\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:27:41.778426 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:41.778395    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-tmp-dir\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:27:41.778426 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:41.778404    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-model-cache\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:27:41.780094 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:41.780066    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-kube-api-access-kbvvk" (OuterVolumeSpecName: "kube-api-access-kbvvk") pod "7c748f9e-70e0-42ba-a00f-aa9b38dcddd2" (UID: "7c748f9e-70e0-42ba-a00f-aa9b38dcddd2"). InnerVolumeSpecName "kube-api-access-kbvvk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:27:41.780094 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:41.780069    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-dshm" (OuterVolumeSpecName: "dshm") pod "7c748f9e-70e0-42ba-a00f-aa9b38dcddd2" (UID: "7c748f9e-70e0-42ba-a00f-aa9b38dcddd2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:27:41.780254 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:41.780106    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7c748f9e-70e0-42ba-a00f-aa9b38dcddd2" (UID: "7c748f9e-70e0-42ba-a00f-aa9b38dcddd2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:27:41.879206 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:41.879174    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbvvk\" (UniqueName: \"kubernetes.io/projected/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-kube-api-access-kbvvk\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:27:41.879206 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:41.879201    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-dshm\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:27:41.879206 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:41.879211    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2-tls-certs\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:27:42.126363 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:42.126324    2570 generic.go:356] "Generic (PLEG): container finished" podID="7c748f9e-70e0-42ba-a00f-aa9b38dcddd2" containerID="f66a9731e5bd99bd864e25936692a1448d604c3744df59df53d1a62e62371212" exitCode=0
Jul 26 18:27:42.126542 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:42.126370    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"
Jul 26 18:27:42.126542 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:42.126416    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf" event={"ID":"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2","Type":"ContainerDied","Data":"f66a9731e5bd99bd864e25936692a1448d604c3744df59df53d1a62e62371212"}
Jul 26 18:27:42.126542 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:42.126456    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf" event={"ID":"7c748f9e-70e0-42ba-a00f-aa9b38dcddd2","Type":"ContainerDied","Data":"0b683b653e34eaf22bb0e4b16a9330ee5b33a3b85b250b9297f7adf20d499587"}
Jul 26 18:27:42.126542 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:42.126473    2570 scope.go:117] "RemoveContainer" containerID="f66a9731e5bd99bd864e25936692a1448d604c3744df59df53d1a62e62371212"
Jul 26 18:27:42.135503 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:42.135477    2570 scope.go:117] "RemoveContainer" containerID="f66a9731e5bd99bd864e25936692a1448d604c3744df59df53d1a62e62371212"
Jul 26 18:27:42.135771 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:27:42.135747    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f66a9731e5bd99bd864e25936692a1448d604c3744df59df53d1a62e62371212\": container with ID starting with f66a9731e5bd99bd864e25936692a1448d604c3744df59df53d1a62e62371212 not found: ID does not exist" containerID="f66a9731e5bd99bd864e25936692a1448d604c3744df59df53d1a62e62371212"
Jul 26 18:27:42.135848 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:42.135779    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f66a9731e5bd99bd864e25936692a1448d604c3744df59df53d1a62e62371212"} err="failed to get container status \"f66a9731e5bd99bd864e25936692a1448d604c3744df59df53d1a62e62371212\": rpc error: code = NotFound desc = could not find container \"f66a9731e5bd99bd864e25936692a1448d604c3744df59df53d1a62e62371212\": container with ID starting with f66a9731e5bd99bd864e25936692a1448d604c3744df59df53d1a62e62371212 not found: ID does not exist"
Jul 26 18:27:42.142947 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:42.142922    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"]
Jul 26 18:27:42.146616 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:42.146595    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-40c67c/test-model-40c67c-kserve-568f8b775-wbvhf"]
Jul 26 18:27:43.839009 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:43.838972    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c748f9e-70e0-42ba-a00f-aa9b38dcddd2" path="/var/lib/kubelet/pods/7c748f9e-70e0-42ba-a00f-aa9b38dcddd2/volumes"
Jul 26 18:27:48.459131 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:48.459088    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-40c67c-8xprl"]
Jul 26 18:27:48.459526 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:48.459460    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c748f9e-70e0-42ba-a00f-aa9b38dcddd2" containerName="main"
Jul 26 18:27:48.459526 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:48.459470    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c748f9e-70e0-42ba-a00f-aa9b38dcddd2" containerName="main"
Jul 26 18:27:48.459526 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:48.459480    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a266d3e-f07a-49f7-b84f-8345273ab92d" containerName="payload-processing"
Jul 26 18:27:48.459526 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:48.459486    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a266d3e-f07a-49f7-b84f-8345273ab92d" containerName="payload-processing"
Jul 26 18:27:48.459526 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:48.459506    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52a8494c-909b-4a5a-85fe-794e61d46fa9" containerName="maas-api"
Jul 26 18:27:48.459526 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:48.459511    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="52a8494c-909b-4a5a-85fe-794e61d46fa9" containerName="maas-api"
Jul 26 18:27:48.459717 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:48.459570    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c748f9e-70e0-42ba-a00f-aa9b38dcddd2" containerName="main"
Jul 26 18:27:48.459717 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:48.459578    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a266d3e-f07a-49f7-b84f-8345273ab92d" containerName="payload-processing"
Jul 26 18:27:48.459717 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:48.459586    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="52a8494c-909b-4a5a-85fe-794e61d46fa9" containerName="maas-api"
Jul 26 18:27:48.463838 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:48.463820    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-40c67c-8xprl"
Jul 26 18:27:48.465943 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:48.465915    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-swzk5\""
Jul 26 18:27:48.469399 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:48.469375    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-40c67c-8xprl"]
Jul 26 18:27:48.536105 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:48.536060    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a4c2b14-0fc8-47cf-a07d-fbc76501bd2d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-40c67c-8xprl\" (UID: \"3a4c2b14-0fc8-47cf-a07d-fbc76501bd2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-40c67c-8xprl"
Jul 26 18:27:48.637559 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:48.637518    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a4c2b14-0fc8-47cf-a07d-fbc76501bd2d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-40c67c-8xprl\" (UID: \"3a4c2b14-0fc8-47cf-a07d-fbc76501bd2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-40c67c-8xprl"
Jul 26 18:27:48.638171 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:48.638153    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a4c2b14-0fc8-47cf-a07d-fbc76501bd2d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-40c67c-8xprl\" (UID: \"3a4c2b14-0fc8-47cf-a07d-fbc76501bd2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-40c67c-8xprl"
Jul 26 18:27:48.775943 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:48.775913    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-40c67c-8xprl"
Jul 26 18:27:48.908405 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:48.908373    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-40c67c-8xprl"]
Jul 26 18:27:48.909666 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:27:48.909639    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a4c2b14_0fc8_47cf_a07d_fbc76501bd2d.slice/crio-f99bbca020384f57a9b3ffc172bcd6d18c5cefb821e2be2466e9e23f848badc1 WatchSource:0}: Error finding container f99bbca020384f57a9b3ffc172bcd6d18c5cefb821e2be2466e9e23f848badc1: Status 404 returned error can't find the container with id f99bbca020384f57a9b3ffc172bcd6d18c5cefb821e2be2466e9e23f848badc1
Jul 26 18:27:49.164736 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:49.164703    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-40c67c-8xprl" event={"ID":"3a4c2b14-0fc8-47cf-a07d-fbc76501bd2d","Type":"ContainerStarted","Data":"f99bbca020384f57a9b3ffc172bcd6d18c5cefb821e2be2466e9e23f848badc1"}
Jul 26 18:27:50.171448 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:50.171413    2570 generic.go:356] "Generic (PLEG): container finished" podID="3a4c2b14-0fc8-47cf-a07d-fbc76501bd2d" containerID="a00b92d057034534ddce69dd3966d75fe08e73fa423de1ae82bbbce70f26dc52" exitCode=0
Jul 26 18:27:50.171871 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:50.171496    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-40c67c-8xprl" event={"ID":"3a4c2b14-0fc8-47cf-a07d-fbc76501bd2d","Type":"ContainerDied","Data":"a00b92d057034534ddce69dd3966d75fe08e73fa423de1ae82bbbce70f26dc52"}
Jul 26 18:27:51.305285 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:51.305252    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-40c67c-8xprl"
Jul 26 18:27:51.465437 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:51.465338    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a4c2b14-0fc8-47cf-a07d-fbc76501bd2d-maas-api-service-ca\") pod \"3a4c2b14-0fc8-47cf-a07d-fbc76501bd2d\" (UID: \"3a4c2b14-0fc8-47cf-a07d-fbc76501bd2d\") "
Jul 26 18:27:51.465694 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:51.465671    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a4c2b14-0fc8-47cf-a07d-fbc76501bd2d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3a4c2b14-0fc8-47cf-a07d-fbc76501bd2d" (UID: "3a4c2b14-0fc8-47cf-a07d-fbc76501bd2d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:27:51.566491 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:51.566454    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a4c2b14-0fc8-47cf-a07d-fbc76501bd2d-maas-api-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:27:52.184143 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:52.184029    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-40c67c-8xprl" event={"ID":"3a4c2b14-0fc8-47cf-a07d-fbc76501bd2d","Type":"ContainerDied","Data":"f99bbca020384f57a9b3ffc172bcd6d18c5cefb821e2be2466e9e23f848badc1"}
Jul 26 18:27:52.184143 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:52.184061    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-40c67c-8xprl"
Jul 26 18:27:52.184143 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:52.184081    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f99bbca020384f57a9b3ffc172bcd6d18c5cefb821e2be2466e9e23f848badc1"
Jul 26 18:27:58.864561 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:58.864478    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2"]
Jul 26 18:27:58.867188 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:58.864688    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2" podUID="209496dd-c721-4ea2-adc6-a8a3e58e831d" containerName="maas-api" containerID="cri-o://94a8c8848a804e15eb59f08f127cac0113bb9efe78d1bbf9458d85ae18c68a2c" gracePeriod=30
Jul 26 18:27:58.922995 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:58.922956    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68"]
Jul 26 18:27:58.923240 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:58.923211    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68" podUID="2aced937-5fc2-4333-8358-339e40a8862a" containerName="payload-pre-processing" containerID="cri-o://06b96bcf3fa290fcc5a6431dc7db1a68af79ddd4116f0ebf640cf5e5e921dafc" gracePeriod=30
Jul 26 18:27:59.170074 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.170047    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2"
Jul 26 18:27:59.196931 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.196905    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68"
Jul 26 18:27:59.222492 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.222464    2570 generic.go:356] "Generic (PLEG): container finished" podID="209496dd-c721-4ea2-adc6-a8a3e58e831d" containerID="94a8c8848a804e15eb59f08f127cac0113bb9efe78d1bbf9458d85ae18c68a2c" exitCode=0
Jul 26 18:27:59.222671 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.222502    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2"
Jul 26 18:27:59.222671 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.222553    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2" event={"ID":"209496dd-c721-4ea2-adc6-a8a3e58e831d","Type":"ContainerDied","Data":"94a8c8848a804e15eb59f08f127cac0113bb9efe78d1bbf9458d85ae18c68a2c"}
Jul 26 18:27:59.222671 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.222607    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2" event={"ID":"209496dd-c721-4ea2-adc6-a8a3e58e831d","Type":"ContainerDied","Data":"5b045bebd995e2881e86cc28aabe6584e8901b7ea781c8aa3c0afacc4504595d"}
Jul 26 18:27:59.222671 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.222632    2570 scope.go:117] "RemoveContainer" containerID="94a8c8848a804e15eb59f08f127cac0113bb9efe78d1bbf9458d85ae18c68a2c"
Jul 26 18:27:59.223847 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.223826    2570 generic.go:356] "Generic (PLEG): container finished" podID="2aced937-5fc2-4333-8358-339e40a8862a" containerID="06b96bcf3fa290fcc5a6431dc7db1a68af79ddd4116f0ebf640cf5e5e921dafc" exitCode=0
Jul 26 18:27:59.223945 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.223874    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68"
Jul 26 18:27:59.223945 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.223905    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68" event={"ID":"2aced937-5fc2-4333-8358-339e40a8862a","Type":"ContainerDied","Data":"06b96bcf3fa290fcc5a6431dc7db1a68af79ddd4116f0ebf640cf5e5e921dafc"}
Jul 26 18:27:59.224060 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.223951    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68" event={"ID":"2aced937-5fc2-4333-8358-339e40a8862a","Type":"ContainerDied","Data":"6fa9bb3587238e11f2969422f0db3173bb5d10c22543e498ddc0c6977c458ab6"}
Jul 26 18:27:59.234661 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.233516    2570 scope.go:117] "RemoveContainer" containerID="94a8c8848a804e15eb59f08f127cac0113bb9efe78d1bbf9458d85ae18c68a2c"
Jul 26 18:27:59.234661 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:27:59.233869    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94a8c8848a804e15eb59f08f127cac0113bb9efe78d1bbf9458d85ae18c68a2c\": container with ID starting with 94a8c8848a804e15eb59f08f127cac0113bb9efe78d1bbf9458d85ae18c68a2c not found: ID does not exist" containerID="94a8c8848a804e15eb59f08f127cac0113bb9efe78d1bbf9458d85ae18c68a2c"
Jul 26 18:27:59.234661 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.233900    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94a8c8848a804e15eb59f08f127cac0113bb9efe78d1bbf9458d85ae18c68a2c"} err="failed to get container status \"94a8c8848a804e15eb59f08f127cac0113bb9efe78d1bbf9458d85ae18c68a2c\": rpc error: code = NotFound desc = could not find container \"94a8c8848a804e15eb59f08f127cac0113bb9efe78d1bbf9458d85ae18c68a2c\": container with ID starting with 94a8c8848a804e15eb59f08f127cac0113bb9efe78d1bbf9458d85ae18c68a2c not found: ID does not exist"
Jul 26 18:27:59.234661 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.233923    2570 scope.go:117] "RemoveContainer" containerID="06b96bcf3fa290fcc5a6431dc7db1a68af79ddd4116f0ebf640cf5e5e921dafc"
Jul 26 18:27:59.234925 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.234890    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/209496dd-c721-4ea2-adc6-a8a3e58e831d-openshift-service-ca\") pod \"209496dd-c721-4ea2-adc6-a8a3e58e831d\" (UID: \"209496dd-c721-4ea2-adc6-a8a3e58e831d\") "
Jul 26 18:27:59.234986 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.234924    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/209496dd-c721-4ea2-adc6-a8a3e58e831d-maas-api-tls\") pod \"209496dd-c721-4ea2-adc6-a8a3e58e831d\" (UID: \"209496dd-c721-4ea2-adc6-a8a3e58e831d\") "
Jul 26 18:27:59.234986 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.234958    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2aced937-5fc2-4333-8358-339e40a8862a-plugins-config-volume\") pod \"2aced937-5fc2-4333-8358-339e40a8862a\" (UID: \"2aced937-5fc2-4333-8358-339e40a8862a\") "
Jul 26 18:27:59.235096 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.235020    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdm2l\" (UniqueName: \"kubernetes.io/projected/2aced937-5fc2-4333-8358-339e40a8862a-kube-api-access-zdm2l\") pod \"2aced937-5fc2-4333-8358-339e40a8862a\" (UID: \"2aced937-5fc2-4333-8358-339e40a8862a\") "
Jul 26 18:27:59.235096 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.235056    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lz4k9\" (UniqueName: \"kubernetes.io/projected/209496dd-c721-4ea2-adc6-a8a3e58e831d-kube-api-access-lz4k9\") pod \"209496dd-c721-4ea2-adc6-a8a3e58e831d\" (UID: \"209496dd-c721-4ea2-adc6-a8a3e58e831d\") "
Jul 26 18:27:59.235688 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.235449    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2aced937-5fc2-4333-8358-339e40a8862a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2aced937-5fc2-4333-8358-339e40a8862a" (UID: "2aced937-5fc2-4333-8358-339e40a8862a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:27:59.235688 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.235487    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/209496dd-c721-4ea2-adc6-a8a3e58e831d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "209496dd-c721-4ea2-adc6-a8a3e58e831d" (UID: "209496dd-c721-4ea2-adc6-a8a3e58e831d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:27:59.235887 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.235764    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/209496dd-c721-4ea2-adc6-a8a3e58e831d-openshift-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:27:59.235887 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.235787    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2aced937-5fc2-4333-8358-339e40a8862a-plugins-config-volume\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:27:59.237710 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.237674    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2aced937-5fc2-4333-8358-339e40a8862a-kube-api-access-zdm2l" (OuterVolumeSpecName: "kube-api-access-zdm2l") pod "2aced937-5fc2-4333-8358-339e40a8862a" (UID: "2aced937-5fc2-4333-8358-339e40a8862a"). InnerVolumeSpecName "kube-api-access-zdm2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:27:59.237801 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.237722    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/209496dd-c721-4ea2-adc6-a8a3e58e831d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "209496dd-c721-4ea2-adc6-a8a3e58e831d" (UID: "209496dd-c721-4ea2-adc6-a8a3e58e831d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:27:59.237801 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.237752    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/209496dd-c721-4ea2-adc6-a8a3e58e831d-kube-api-access-lz4k9" (OuterVolumeSpecName: "kube-api-access-lz4k9") pod "209496dd-c721-4ea2-adc6-a8a3e58e831d" (UID: "209496dd-c721-4ea2-adc6-a8a3e58e831d"). InnerVolumeSpecName "kube-api-access-lz4k9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:27:59.244067 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.244050    2570 scope.go:117] "RemoveContainer" containerID="06b96bcf3fa290fcc5a6431dc7db1a68af79ddd4116f0ebf640cf5e5e921dafc"
Jul 26 18:27:59.244410 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:27:59.244380    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06b96bcf3fa290fcc5a6431dc7db1a68af79ddd4116f0ebf640cf5e5e921dafc\": container with ID starting with 06b96bcf3fa290fcc5a6431dc7db1a68af79ddd4116f0ebf640cf5e5e921dafc not found: ID does not exist" containerID="06b96bcf3fa290fcc5a6431dc7db1a68af79ddd4116f0ebf640cf5e5e921dafc"
Jul 26 18:27:59.244495 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.244408    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06b96bcf3fa290fcc5a6431dc7db1a68af79ddd4116f0ebf640cf5e5e921dafc"} err="failed to get container status \"06b96bcf3fa290fcc5a6431dc7db1a68af79ddd4116f0ebf640cf5e5e921dafc\": rpc error: code = NotFound desc = could not find container \"06b96bcf3fa290fcc5a6431dc7db1a68af79ddd4116f0ebf640cf5e5e921dafc\": container with ID starting with 06b96bcf3fa290fcc5a6431dc7db1a68af79ddd4116f0ebf640cf5e5e921dafc not found: ID does not exist"
Jul 26 18:27:59.336454 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.336421    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/209496dd-c721-4ea2-adc6-a8a3e58e831d-maas-api-tls\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:27:59.336454 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.336451    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdm2l\" (UniqueName: \"kubernetes.io/projected/2aced937-5fc2-4333-8358-339e40a8862a-kube-api-access-zdm2l\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:27:59.336454 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.336463    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lz4k9\" (UniqueName: \"kubernetes.io/projected/209496dd-c721-4ea2-adc6-a8a3e58e831d-kube-api-access-lz4k9\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:27:59.547156 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.547104    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2"]
Jul 26 18:27:59.549433 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.549407    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-40c67c-749764cc8b-cdwt2"]
Jul 26 18:27:59.556956 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.556931    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68"]
Jul 26 18:27:59.559763 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.559740    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-40c67c-76448c8648-5qv68"]
Jul 26 18:27:59.839080 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.839002    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="209496dd-c721-4ea2-adc6-a8a3e58e831d" path="/var/lib/kubelet/pods/209496dd-c721-4ea2-adc6-a8a3e58e831d/volumes"
Jul 26 18:27:59.839408 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:27:59.839394    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2aced937-5fc2-4333-8358-339e40a8862a" path="/var/lib/kubelet/pods/2aced937-5fc2-4333-8358-339e40a8862a/volumes"
Jul 26 18:28:03.530366 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:03.530329    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-40c67c-8xprl"]
Jul 26 18:28:03.532736 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:03.532712    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-40c67c-8xprl"]
Jul 26 18:28:03.838045 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:03.837966    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a4c2b14-0fc8-47cf-a07d-fbc76501bd2d" path="/var/lib/kubelet/pods/3a4c2b14-0fc8-47cf-a07d-fbc76501bd2d/volumes"
Jul 26 18:28:14.167828 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.167793    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr"]
Jul 26 18:28:14.168247 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.168174    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="209496dd-c721-4ea2-adc6-a8a3e58e831d" containerName="maas-api"
Jul 26 18:28:14.168247 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.168187    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="209496dd-c721-4ea2-adc6-a8a3e58e831d" containerName="maas-api"
Jul 26 18:28:14.168247 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.168195    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2aced937-5fc2-4333-8358-339e40a8862a" containerName="payload-pre-processing"
Jul 26 18:28:14.168247 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.168202    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2aced937-5fc2-4333-8358-339e40a8862a" containerName="payload-pre-processing"
Jul 26 18:28:14.168247 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.168219    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a4c2b14-0fc8-47cf-a07d-fbc76501bd2d" containerName="revoke-keys"
Jul 26 18:28:14.168247 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.168225    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a4c2b14-0fc8-47cf-a07d-fbc76501bd2d" containerName="revoke-keys"
Jul 26 18:28:14.168458 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.168281    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="209496dd-c721-4ea2-adc6-a8a3e58e831d" containerName="maas-api"
Jul 26 18:28:14.168458 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.168293    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2aced937-5fc2-4333-8358-339e40a8862a" containerName="payload-pre-processing"
Jul 26 18:28:14.168458 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.168300    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a4c2b14-0fc8-47cf-a07d-fbc76501bd2d" containerName="revoke-keys"
Jul 26 18:28:14.171404 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.171381    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr"
Jul 26 18:28:14.173456 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.173433    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-b6e870-serving-cert\""
Jul 26 18:28:14.182206 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.182183    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr"]
Jul 26 18:28:14.270228 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.270193    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/614c8228-77ca-491b-aac2-0737f67514ff-maas-api-tls\") pod \"maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr\" (UID: \"614c8228-77ca-491b-aac2-0737f67514ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr"
Jul 26 18:28:14.270411 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.270234    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jhfk\" (UniqueName: \"kubernetes.io/projected/614c8228-77ca-491b-aac2-0737f67514ff-kube-api-access-7jhfk\") pod \"maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr\" (UID: \"614c8228-77ca-491b-aac2-0737f67514ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr"
Jul 26 18:28:14.270411 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.270351    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/614c8228-77ca-491b-aac2-0737f67514ff-openshift-service-ca\") pod \"maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr\" (UID: \"614c8228-77ca-491b-aac2-0737f67514ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr"
Jul 26 18:28:14.371510 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.371472    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/614c8228-77ca-491b-aac2-0737f67514ff-openshift-service-ca\") pod \"maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr\" (UID: \"614c8228-77ca-491b-aac2-0737f67514ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr"
Jul 26 18:28:14.371692 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.371533    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/614c8228-77ca-491b-aac2-0737f67514ff-maas-api-tls\") pod \"maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr\" (UID: \"614c8228-77ca-491b-aac2-0737f67514ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr"
Jul 26 18:28:14.371692 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.371567    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jhfk\" (UniqueName: \"kubernetes.io/projected/614c8228-77ca-491b-aac2-0737f67514ff-kube-api-access-7jhfk\") pod \"maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr\" (UID: \"614c8228-77ca-491b-aac2-0737f67514ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr"
Jul 26 18:28:14.372264 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.372241    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/614c8228-77ca-491b-aac2-0737f67514ff-openshift-service-ca\") pod \"maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr\" (UID: \"614c8228-77ca-491b-aac2-0737f67514ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr"
Jul 26 18:28:14.374042 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.374019    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/614c8228-77ca-491b-aac2-0737f67514ff-maas-api-tls\") pod \"maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr\" (UID: \"614c8228-77ca-491b-aac2-0737f67514ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr"
Jul 26 18:28:14.379990 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.379966    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jhfk\" (UniqueName: \"kubernetes.io/projected/614c8228-77ca-491b-aac2-0737f67514ff-kube-api-access-7jhfk\") pod \"maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr\" (UID: \"614c8228-77ca-491b-aac2-0737f67514ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr"
Jul 26 18:28:14.482388 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.482290    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr"
Jul 26 18:28:14.619734 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.619703    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr"]
Jul 26 18:28:14.622016 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:28:14.621988    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod614c8228_77ca_491b_aac2_0737f67514ff.slice/crio-8459516c972d95b5e56bd5a7d34e33bebac1093f4f1d303c0d581e5c075e0d44 WatchSource:0}: Error finding container 8459516c972d95b5e56bd5a7d34e33bebac1093f4f1d303c0d581e5c075e0d44: Status 404 returned error can't find the container with id 8459516c972d95b5e56bd5a7d34e33bebac1093f4f1d303c0d581e5c075e0d44
Jul 26 18:28:14.623363 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:14.623341    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 18:28:15.313203 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:15.313169    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr" event={"ID":"614c8228-77ca-491b-aac2-0737f67514ff","Type":"ContainerStarted","Data":"f264fc3efc2ff6a3b456a6d7daffceefe434ee36624994c2b9fa6069d72328e1"}
Jul 26 18:28:15.313616 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:15.313232    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr" event={"ID":"614c8228-77ca-491b-aac2-0737f67514ff","Type":"ContainerStarted","Data":"8459516c972d95b5e56bd5a7d34e33bebac1093f4f1d303c0d581e5c075e0d44"}
Jul 26 18:28:15.313616 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:15.313261    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr"
Jul 26 18:28:15.333070 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:15.333017    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr" podStartSLOduration=0.853984082 podStartE2EDuration="1.332999392s" totalImagesPullingTime="479.01531ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:28:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:28:15.330880853 +0000 UTC m=+2351.983071712" watchObservedRunningTime="2026-07-26 18:28:15.332999392 +0000 UTC m=+2351.985190249"
Jul 26 18:28:21.323614 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:21.323584    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr"
Jul 26 18:28:22.273817 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:22.273783    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6"]
Jul 26 18:28:22.278231 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:22.278203    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6"
Jul 26 18:28:22.281206 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:22.281180    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-167468-serving-cert\""
Jul 26 18:28:22.293378 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:22.293344    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6"]
Jul 26 18:28:22.347108 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:22.347072    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3-maas-api-tls\") pod \"maas-api-e2e-api-b-167468-76b4c889b6-mx5d6\" (UID: \"fd203e05-a7e9-4d9a-9667-c3c2338f9ee3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6"
Jul 26 18:28:22.347525 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:22.347194    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vdrm\" (UniqueName: \"kubernetes.io/projected/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3-kube-api-access-9vdrm\") pod \"maas-api-e2e-api-b-167468-76b4c889b6-mx5d6\" (UID: \"fd203e05-a7e9-4d9a-9667-c3c2338f9ee3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6"
Jul 26 18:28:22.347525 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:22.347236    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3-openshift-service-ca\") pod \"maas-api-e2e-api-b-167468-76b4c889b6-mx5d6\" (UID: \"fd203e05-a7e9-4d9a-9667-c3c2338f9ee3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6"
Jul 26 18:28:22.448243 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:22.448198    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3-maas-api-tls\") pod \"maas-api-e2e-api-b-167468-76b4c889b6-mx5d6\" (UID: \"fd203e05-a7e9-4d9a-9667-c3c2338f9ee3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6"
Jul 26 18:28:22.448431 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:22.448267    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vdrm\" (UniqueName: \"kubernetes.io/projected/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3-kube-api-access-9vdrm\") pod \"maas-api-e2e-api-b-167468-76b4c889b6-mx5d6\" (UID: \"fd203e05-a7e9-4d9a-9667-c3c2338f9ee3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6"
Jul 26 18:28:22.448431 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:22.448296    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3-openshift-service-ca\") pod \"maas-api-e2e-api-b-167468-76b4c889b6-mx5d6\" (UID: \"fd203e05-a7e9-4d9a-9667-c3c2338f9ee3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6"
Jul 26 18:28:22.448431 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:28:22.448388    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-167468-serving-cert: secret "maas-api-e2e-api-b-167468-serving-cert" not found
Jul 26 18:28:22.448546 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:28:22.448467    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3-maas-api-tls podName:fd203e05-a7e9-4d9a-9667-c3c2338f9ee3 nodeName:}" failed. No retries permitted until 2026-07-26 18:28:22.948446902 +0000 UTC m=+2359.600637738 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3-maas-api-tls") pod "maas-api-e2e-api-b-167468-76b4c889b6-mx5d6" (UID: "fd203e05-a7e9-4d9a-9667-c3c2338f9ee3") : secret "maas-api-e2e-api-b-167468-serving-cert" not found
Jul 26 18:28:22.448925 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:22.448907    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3-openshift-service-ca\") pod \"maas-api-e2e-api-b-167468-76b4c889b6-mx5d6\" (UID: \"fd203e05-a7e9-4d9a-9667-c3c2338f9ee3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6"
Jul 26 18:28:22.456813 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:22.456784    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vdrm\" (UniqueName: \"kubernetes.io/projected/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3-kube-api-access-9vdrm\") pod \"maas-api-e2e-api-b-167468-76b4c889b6-mx5d6\" (UID: \"fd203e05-a7e9-4d9a-9667-c3c2338f9ee3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6"
Jul 26 18:28:22.953482 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:22.953445    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3-maas-api-tls\") pod \"maas-api-e2e-api-b-167468-76b4c889b6-mx5d6\" (UID: \"fd203e05-a7e9-4d9a-9667-c3c2338f9ee3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6"
Jul 26 18:28:22.953702 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:28:22.953629    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-167468-serving-cert: secret "maas-api-e2e-api-b-167468-serving-cert" not found
Jul 26 18:28:22.953768 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:28:22.953714    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3-maas-api-tls podName:fd203e05-a7e9-4d9a-9667-c3c2338f9ee3 nodeName:}" failed. No retries permitted until 2026-07-26 18:28:23.953691489 +0000 UTC m=+2360.605882340 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3-maas-api-tls") pod "maas-api-e2e-api-b-167468-76b4c889b6-mx5d6" (UID: "fd203e05-a7e9-4d9a-9667-c3c2338f9ee3") : secret "maas-api-e2e-api-b-167468-serving-cert" not found
Jul 26 18:28:23.962244 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:23.962208    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3-maas-api-tls\") pod \"maas-api-e2e-api-b-167468-76b4c889b6-mx5d6\" (UID: \"fd203e05-a7e9-4d9a-9667-c3c2338f9ee3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6"
Jul 26 18:28:23.964763 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:23.964735    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3-maas-api-tls\") pod \"maas-api-e2e-api-b-167468-76b4c889b6-mx5d6\" (UID: \"fd203e05-a7e9-4d9a-9667-c3c2338f9ee3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6"
Jul 26 18:28:24.091804 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:24.091768    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6"
Jul 26 18:28:24.253018 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:24.252985    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6"]
Jul 26 18:28:24.254399 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:28:24.254369    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd203e05_a7e9_4d9a_9667_c3c2338f9ee3.slice/crio-405070a46e60941c7df263e971e6622b2594da5cc3f5bf18539fc862ca1a99fa WatchSource:0}: Error finding container 405070a46e60941c7df263e971e6622b2594da5cc3f5bf18539fc862ca1a99fa: Status 404 returned error can't find the container with id 405070a46e60941c7df263e971e6622b2594da5cc3f5bf18539fc862ca1a99fa
Jul 26 18:28:24.363721 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:24.363688    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6" event={"ID":"fd203e05-a7e9-4d9a-9667-c3c2338f9ee3","Type":"ContainerStarted","Data":"405070a46e60941c7df263e971e6622b2594da5cc3f5bf18539fc862ca1a99fa"}
Jul 26 18:28:25.371965 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:25.371928    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6" event={"ID":"fd203e05-a7e9-4d9a-9667-c3c2338f9ee3","Type":"ContainerStarted","Data":"48339d56e9d6ba0a85f331b76d72d5b7be3ebd95eebc860abe8ab0f14e846edd"}
Jul 26 18:28:25.372366 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:25.371992    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6"
Jul 26 18:28:25.390056 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:25.390003    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6" podStartSLOduration=3.012701749 podStartE2EDuration="3.389988628s" totalImagesPullingTime="377.286879ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:28:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:28:25.387814899 +0000 UTC m=+2362.040005758" watchObservedRunningTime="2026-07-26 18:28:25.389988628 +0000 UTC m=+2362.042179486"
Jul 26 18:28:31.382497 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:31.382465    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6"
Jul 26 18:28:34.497266 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:34.497232    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6e870-n4t7w"]
Jul 26 18:28:34.501079 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:34.501062    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6e870-n4t7w"
Jul 26 18:28:34.503187 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:34.503157    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-swzk5\""
Jul 26 18:28:34.513428 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:34.511941    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6e870-n4t7w"]
Jul 26 18:28:34.658083 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:34.658039    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f82a2461-992f-4503-836b-1eab884cd636-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b6e870-n4t7w\" (UID: \"f82a2461-992f-4503-836b-1eab884cd636\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6e870-n4t7w"
Jul 26 18:28:34.758964 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:34.758922    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f82a2461-992f-4503-836b-1eab884cd636-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b6e870-n4t7w\" (UID: \"f82a2461-992f-4503-836b-1eab884cd636\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6e870-n4t7w"
Jul 26 18:28:34.759591 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:34.759569    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f82a2461-992f-4503-836b-1eab884cd636-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b6e870-n4t7w\" (UID: \"f82a2461-992f-4503-836b-1eab884cd636\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6e870-n4t7w"
Jul 26 18:28:34.819676 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:34.819642    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6e870-n4t7w"
Jul 26 18:28:34.958022 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:34.957990    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6e870-n4t7w"]
Jul 26 18:28:34.959461 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:28:34.959431    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf82a2461_992f_4503_836b_1eab884cd636.slice/crio-6e0eee2125a2a6a427e27f005e6151f335b4589f93ca75b63aac067bf47a5c2e WatchSource:0}: Error finding container 6e0eee2125a2a6a427e27f005e6151f335b4589f93ca75b63aac067bf47a5c2e: Status 404 returned error can't find the container with id 6e0eee2125a2a6a427e27f005e6151f335b4589f93ca75b63aac067bf47a5c2e
Jul 26 18:28:35.427777 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:35.427673    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6e870-n4t7w" event={"ID":"f82a2461-992f-4503-836b-1eab884cd636","Type":"ContainerStarted","Data":"e1a7aa0f11626bf7405bb45b1303892826d004fb2a4d7a66fabd148a96cc388b"}
Jul 26 18:28:35.427777 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:35.427733    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6e870-n4t7w" event={"ID":"f82a2461-992f-4503-836b-1eab884cd636","Type":"ContainerStarted","Data":"6e0eee2125a2a6a427e27f005e6151f335b4589f93ca75b63aac067bf47a5c2e"}
Jul 26 18:28:35.445099 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:35.445049    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6e870-n4t7w" podStartSLOduration=1.445029317 podStartE2EDuration="1.445029317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:28:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:28:35.443266428 +0000 UTC m=+2372.095457289" watchObservedRunningTime="2026-07-26 18:28:35.445029317 +0000 UTC m=+2372.097220176"
Jul 26 18:28:36.434841 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:36.434807    2570 generic.go:356] "Generic (PLEG): container finished" podID="f82a2461-992f-4503-836b-1eab884cd636" containerID="e1a7aa0f11626bf7405bb45b1303892826d004fb2a4d7a66fabd148a96cc388b" exitCode=0
Jul 26 18:28:36.435259 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:36.434855    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6e870-n4t7w" event={"ID":"f82a2461-992f-4503-836b-1eab884cd636","Type":"ContainerDied","Data":"e1a7aa0f11626bf7405bb45b1303892826d004fb2a4d7a66fabd148a96cc388b"}
Jul 26 18:28:37.575272 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:37.575249    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6e870-n4t7w"
Jul 26 18:28:37.682877 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:37.682838    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f82a2461-992f-4503-836b-1eab884cd636-maas-api-service-ca\") pod \"f82a2461-992f-4503-836b-1eab884cd636\" (UID: \"f82a2461-992f-4503-836b-1eab884cd636\") "
Jul 26 18:28:37.683247 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:37.683225    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f82a2461-992f-4503-836b-1eab884cd636-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f82a2461-992f-4503-836b-1eab884cd636" (UID: "f82a2461-992f-4503-836b-1eab884cd636"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:28:37.784365 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:37.784326    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f82a2461-992f-4503-836b-1eab884cd636-maas-api-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:28:38.446235 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:38.446201    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6e870-n4t7w" event={"ID":"f82a2461-992f-4503-836b-1eab884cd636","Type":"ContainerDied","Data":"6e0eee2125a2a6a427e27f005e6151f335b4589f93ca75b63aac067bf47a5c2e"}
Jul 26 18:28:38.446235 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:38.446228    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6e870-n4t7w"
Jul 26 18:28:38.446451 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:38.446249    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e0eee2125a2a6a427e27f005e6151f335b4589f93ca75b63aac067bf47a5c2e"
Jul 26 18:28:49.691591 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:49.691552    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr"]
Jul 26 18:28:49.692102 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:49.691810    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr" podUID="614c8228-77ca-491b-aac2-0737f67514ff" containerName="maas-api" containerID="cri-o://f264fc3efc2ff6a3b456a6d7daffceefe434ee36624994c2b9fa6069d72328e1" gracePeriod=30
Jul 26 18:28:49.944486 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:49.944430    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr"
Jul 26 18:28:50.093811 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:50.093777    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/614c8228-77ca-491b-aac2-0737f67514ff-maas-api-tls\") pod \"614c8228-77ca-491b-aac2-0737f67514ff\" (UID: \"614c8228-77ca-491b-aac2-0737f67514ff\") "
Jul 26 18:28:50.093991 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:50.093822    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jhfk\" (UniqueName: \"kubernetes.io/projected/614c8228-77ca-491b-aac2-0737f67514ff-kube-api-access-7jhfk\") pod \"614c8228-77ca-491b-aac2-0737f67514ff\" (UID: \"614c8228-77ca-491b-aac2-0737f67514ff\") "
Jul 26 18:28:50.093991 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:50.093873    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/614c8228-77ca-491b-aac2-0737f67514ff-openshift-service-ca\") pod \"614c8228-77ca-491b-aac2-0737f67514ff\" (UID: \"614c8228-77ca-491b-aac2-0737f67514ff\") "
Jul 26 18:28:50.094406 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:50.094377    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/614c8228-77ca-491b-aac2-0737f67514ff-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "614c8228-77ca-491b-aac2-0737f67514ff" (UID: "614c8228-77ca-491b-aac2-0737f67514ff"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:28:50.096081 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:50.096056    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/614c8228-77ca-491b-aac2-0737f67514ff-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "614c8228-77ca-491b-aac2-0737f67514ff" (UID: "614c8228-77ca-491b-aac2-0737f67514ff"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:28:50.096203 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:50.096096    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/614c8228-77ca-491b-aac2-0737f67514ff-kube-api-access-7jhfk" (OuterVolumeSpecName: "kube-api-access-7jhfk") pod "614c8228-77ca-491b-aac2-0737f67514ff" (UID: "614c8228-77ca-491b-aac2-0737f67514ff"). InnerVolumeSpecName "kube-api-access-7jhfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:28:50.195401 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:50.195317    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/614c8228-77ca-491b-aac2-0737f67514ff-openshift-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:28:50.195401 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:50.195348    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/614c8228-77ca-491b-aac2-0737f67514ff-maas-api-tls\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:28:50.195401 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:50.195358    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jhfk\" (UniqueName: \"kubernetes.io/projected/614c8228-77ca-491b-aac2-0737f67514ff-kube-api-access-7jhfk\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:28:50.512781 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:50.512748    2570 generic.go:356] "Generic (PLEG): container finished" podID="614c8228-77ca-491b-aac2-0737f67514ff" containerID="f264fc3efc2ff6a3b456a6d7daffceefe434ee36624994c2b9fa6069d72328e1" exitCode=0
Jul 26 18:28:50.512960 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:50.512787    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr" event={"ID":"614c8228-77ca-491b-aac2-0737f67514ff","Type":"ContainerDied","Data":"f264fc3efc2ff6a3b456a6d7daffceefe434ee36624994c2b9fa6069d72328e1"}
Jul 26 18:28:50.512960 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:50.512798    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr"
Jul 26 18:28:50.512960 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:50.512822    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr" event={"ID":"614c8228-77ca-491b-aac2-0737f67514ff","Type":"ContainerDied","Data":"8459516c972d95b5e56bd5a7d34e33bebac1093f4f1d303c0d581e5c075e0d44"}
Jul 26 18:28:50.512960 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:50.512844    2570 scope.go:117] "RemoveContainer" containerID="f264fc3efc2ff6a3b456a6d7daffceefe434ee36624994c2b9fa6069d72328e1"
Jul 26 18:28:50.524994 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:50.524975    2570 scope.go:117] "RemoveContainer" containerID="f264fc3efc2ff6a3b456a6d7daffceefe434ee36624994c2b9fa6069d72328e1"
Jul 26 18:28:50.525340 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:28:50.525321    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f264fc3efc2ff6a3b456a6d7daffceefe434ee36624994c2b9fa6069d72328e1\": container with ID starting with f264fc3efc2ff6a3b456a6d7daffceefe434ee36624994c2b9fa6069d72328e1 not found: ID does not exist" containerID="f264fc3efc2ff6a3b456a6d7daffceefe434ee36624994c2b9fa6069d72328e1"
Jul 26 18:28:50.525420 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:50.525353    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f264fc3efc2ff6a3b456a6d7daffceefe434ee36624994c2b9fa6069d72328e1"} err="failed to get container status \"f264fc3efc2ff6a3b456a6d7daffceefe434ee36624994c2b9fa6069d72328e1\": rpc error: code = NotFound desc = could not find container \"f264fc3efc2ff6a3b456a6d7daffceefe434ee36624994c2b9fa6069d72328e1\": container with ID starting with f264fc3efc2ff6a3b456a6d7daffceefe434ee36624994c2b9fa6069d72328e1 not found: ID does not exist"
Jul 26 18:28:50.536926 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:50.536902    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr"]
Jul 26 18:28:50.541085 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:50.541063    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b6e870-7449dbb5-bjqqr"]
Jul 26 18:28:51.841329 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:51.839441    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="614c8228-77ca-491b-aac2-0737f67514ff" path="/var/lib/kubelet/pods/614c8228-77ca-491b-aac2-0737f67514ff/volumes"
Jul 26 18:28:54.579833 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:54.579786    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6e870-n4t7w"]
Jul 26 18:28:54.583043 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:54.583019    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6e870-n4t7w"]
Jul 26 18:28:55.838339 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:28:55.838302    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f82a2461-992f-4503-836b-1eab884cd636" path="/var/lib/kubelet/pods/f82a2461-992f-4503-836b-1eab884cd636/volumes"
Jul 26 18:29:01.266475 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:01.266433    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-167468-7n8lb"]
Jul 26 18:29:01.266993 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:01.266973    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f82a2461-992f-4503-836b-1eab884cd636" containerName="revoke-keys"
Jul 26 18:29:01.267062 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:01.266997    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f82a2461-992f-4503-836b-1eab884cd636" containerName="revoke-keys"
Jul 26 18:29:01.267062 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:01.267013    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="614c8228-77ca-491b-aac2-0737f67514ff" containerName="maas-api"
Jul 26 18:29:01.267062 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:01.267021    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="614c8228-77ca-491b-aac2-0737f67514ff" containerName="maas-api"
Jul 26 18:29:01.267267 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:01.267131    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="614c8228-77ca-491b-aac2-0737f67514ff" containerName="maas-api"
Jul 26 18:29:01.267267 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:01.267145    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f82a2461-992f-4503-836b-1eab884cd636" containerName="revoke-keys"
Jul 26 18:29:01.271631 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:01.271610    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-167468-7n8lb"
Jul 26 18:29:01.274085 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:01.274063    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-swzk5\""
Jul 26 18:29:01.278330 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:01.278302    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-167468-7n8lb"]
Jul 26 18:29:01.288241 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:01.288208    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f222359b-28b3-4fc9-872b-15f63efc1d08-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-167468-7n8lb\" (UID: \"f222359b-28b3-4fc9-872b-15f63efc1d08\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-167468-7n8lb"
Jul 26 18:29:01.388738 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:01.388700    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f222359b-28b3-4fc9-872b-15f63efc1d08-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-167468-7n8lb\" (UID: \"f222359b-28b3-4fc9-872b-15f63efc1d08\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-167468-7n8lb"
Jul 26 18:29:01.389380 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:01.389361    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f222359b-28b3-4fc9-872b-15f63efc1d08-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-167468-7n8lb\" (UID: \"f222359b-28b3-4fc9-872b-15f63efc1d08\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-167468-7n8lb"
Jul 26 18:29:01.584087 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:01.584001    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-167468-7n8lb"
Jul 26 18:29:01.721935 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:01.721897    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-167468-7n8lb"]
Jul 26 18:29:01.723181 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:29:01.723154    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf222359b_28b3_4fc9_872b_15f63efc1d08.slice/crio-a18fab51f7ca1c20c0b88e690dcfdf8fbb8a824bdf3e2997991c575e210aa579 WatchSource:0}: Error finding container a18fab51f7ca1c20c0b88e690dcfdf8fbb8a824bdf3e2997991c575e210aa579: Status 404 returned error can't find the container with id a18fab51f7ca1c20c0b88e690dcfdf8fbb8a824bdf3e2997991c575e210aa579
Jul 26 18:29:02.584522 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:02.584487    2570 generic.go:356] "Generic (PLEG): container finished" podID="f222359b-28b3-4fc9-872b-15f63efc1d08" containerID="0f6bfbeb27f14e612067bca2a1290bad57ed92b5468c081a71bd698ad536c07e" exitCode=0
Jul 26 18:29:02.584981 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:02.584574    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-167468-7n8lb" event={"ID":"f222359b-28b3-4fc9-872b-15f63efc1d08","Type":"ContainerDied","Data":"0f6bfbeb27f14e612067bca2a1290bad57ed92b5468c081a71bd698ad536c07e"}
Jul 26 18:29:02.584981 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:02.584640    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-167468-7n8lb" event={"ID":"f222359b-28b3-4fc9-872b-15f63efc1d08","Type":"ContainerStarted","Data":"a18fab51f7ca1c20c0b88e690dcfdf8fbb8a824bdf3e2997991c575e210aa579"}
Jul 26 18:29:03.725724 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:03.725697    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-167468-7n8lb"
Jul 26 18:29:03.808880 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:03.808850    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f222359b-28b3-4fc9-872b-15f63efc1d08-maas-api-service-ca\") pod \"f222359b-28b3-4fc9-872b-15f63efc1d08\" (UID: \"f222359b-28b3-4fc9-872b-15f63efc1d08\") "
Jul 26 18:29:03.809239 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:03.809215    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f222359b-28b3-4fc9-872b-15f63efc1d08-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f222359b-28b3-4fc9-872b-15f63efc1d08" (UID: "f222359b-28b3-4fc9-872b-15f63efc1d08"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:29:03.909966 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:03.909920    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f222359b-28b3-4fc9-872b-15f63efc1d08-maas-api-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:29:04.023716 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:04.023686    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 18:29:04.035498 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:04.035473    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 18:29:04.596621 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:04.596528    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-167468-7n8lb"
Jul 26 18:29:04.596621 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:04.596570    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-167468-7n8lb" event={"ID":"f222359b-28b3-4fc9-872b-15f63efc1d08","Type":"ContainerDied","Data":"a18fab51f7ca1c20c0b88e690dcfdf8fbb8a824bdf3e2997991c575e210aa579"}
Jul 26 18:29:04.596621 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:04.596619    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a18fab51f7ca1c20c0b88e690dcfdf8fbb8a824bdf3e2997991c575e210aa579"
Jul 26 18:29:16.649359 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:16.649318    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6"]
Jul 26 18:29:16.650247 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:16.650208    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6" podUID="fd203e05-a7e9-4d9a-9667-c3c2338f9ee3" containerName="maas-api" containerID="cri-o://48339d56e9d6ba0a85f331b76d72d5b7be3ebd95eebc860abe8ab0f14e846edd" gracePeriod=30
Jul 26 18:29:16.901747 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:16.901682    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6"
Jul 26 18:29:17.024152 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:17.024089    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3-openshift-service-ca\") pod \"fd203e05-a7e9-4d9a-9667-c3c2338f9ee3\" (UID: \"fd203e05-a7e9-4d9a-9667-c3c2338f9ee3\") "
Jul 26 18:29:17.024339 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:17.024259    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3-maas-api-tls\") pod \"fd203e05-a7e9-4d9a-9667-c3c2338f9ee3\" (UID: \"fd203e05-a7e9-4d9a-9667-c3c2338f9ee3\") "
Jul 26 18:29:17.024339 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:17.024288    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vdrm\" (UniqueName: \"kubernetes.io/projected/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3-kube-api-access-9vdrm\") pod \"fd203e05-a7e9-4d9a-9667-c3c2338f9ee3\" (UID: \"fd203e05-a7e9-4d9a-9667-c3c2338f9ee3\") "
Jul 26 18:29:17.024500 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:17.024475    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fd203e05-a7e9-4d9a-9667-c3c2338f9ee3" (UID: "fd203e05-a7e9-4d9a-9667-c3c2338f9ee3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:29:17.024572 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:17.024555    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3-openshift-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:29:17.026410 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:17.026381    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fd203e05-a7e9-4d9a-9667-c3c2338f9ee3" (UID: "fd203e05-a7e9-4d9a-9667-c3c2338f9ee3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:29:17.026410 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:17.026388    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3-kube-api-access-9vdrm" (OuterVolumeSpecName: "kube-api-access-9vdrm") pod "fd203e05-a7e9-4d9a-9667-c3c2338f9ee3" (UID: "fd203e05-a7e9-4d9a-9667-c3c2338f9ee3"). InnerVolumeSpecName "kube-api-access-9vdrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:29:17.125212 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:17.125178    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3-maas-api-tls\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:29:17.125212 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:17.125209    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vdrm\" (UniqueName: \"kubernetes.io/projected/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3-kube-api-access-9vdrm\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:29:17.667656 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:17.667621    2570 generic.go:356] "Generic (PLEG): container finished" podID="fd203e05-a7e9-4d9a-9667-c3c2338f9ee3" containerID="48339d56e9d6ba0a85f331b76d72d5b7be3ebd95eebc860abe8ab0f14e846edd" exitCode=0
Jul 26 18:29:17.668071 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:17.667665    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6"
Jul 26 18:29:17.668071 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:17.667712    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6" event={"ID":"fd203e05-a7e9-4d9a-9667-c3c2338f9ee3","Type":"ContainerDied","Data":"48339d56e9d6ba0a85f331b76d72d5b7be3ebd95eebc860abe8ab0f14e846edd"}
Jul 26 18:29:17.668071 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:17.667761    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6" event={"ID":"fd203e05-a7e9-4d9a-9667-c3c2338f9ee3","Type":"ContainerDied","Data":"405070a46e60941c7df263e971e6622b2594da5cc3f5bf18539fc862ca1a99fa"}
Jul 26 18:29:17.668071 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:17.667785    2570 scope.go:117] "RemoveContainer" containerID="48339d56e9d6ba0a85f331b76d72d5b7be3ebd95eebc860abe8ab0f14e846edd"
Jul 26 18:29:17.677601 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:17.677583    2570 scope.go:117] "RemoveContainer" containerID="48339d56e9d6ba0a85f331b76d72d5b7be3ebd95eebc860abe8ab0f14e846edd"
Jul 26 18:29:17.677849 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:29:17.677828    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48339d56e9d6ba0a85f331b76d72d5b7be3ebd95eebc860abe8ab0f14e846edd\": container with ID starting with 48339d56e9d6ba0a85f331b76d72d5b7be3ebd95eebc860abe8ab0f14e846edd not found: ID does not exist" containerID="48339d56e9d6ba0a85f331b76d72d5b7be3ebd95eebc860abe8ab0f14e846edd"
Jul 26 18:29:17.677899 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:17.677858    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48339d56e9d6ba0a85f331b76d72d5b7be3ebd95eebc860abe8ab0f14e846edd"} err="failed to get container status \"48339d56e9d6ba0a85f331b76d72d5b7be3ebd95eebc860abe8ab0f14e846edd\": rpc error: code = NotFound desc = could not find container \"48339d56e9d6ba0a85f331b76d72d5b7be3ebd95eebc860abe8ab0f14e846edd\": container with ID starting with 48339d56e9d6ba0a85f331b76d72d5b7be3ebd95eebc860abe8ab0f14e846edd not found: ID does not exist"
Jul 26 18:29:17.689092 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:17.689068    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6"]
Jul 26 18:29:17.692680 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:17.692656    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-167468-76b4c889b6-mx5d6"]
Jul 26 18:29:17.838981 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:17.838946    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd203e05-a7e9-4d9a-9667-c3c2338f9ee3" path="/var/lib/kubelet/pods/fd203e05-a7e9-4d9a-9667-c3c2338f9ee3/volumes"
Jul 26 18:29:21.360721 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:21.360683    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-167468-7n8lb"]
Jul 26 18:29:21.363624 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:21.363596    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-167468-7n8lb"]
Jul 26 18:29:21.838631 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:21.838593    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f222359b-28b3-4fc9-872b-15f63efc1d08" path="/var/lib/kubelet/pods/f222359b-28b3-4fc9-872b-15f63efc1d08/volumes"
Jul 26 18:29:31.716464 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:31.716369    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9"]
Jul 26 18:29:31.716937 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:31.716763    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f222359b-28b3-4fc9-872b-15f63efc1d08" containerName="revoke-keys"
Jul 26 18:29:31.716937 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:31.716774    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f222359b-28b3-4fc9-872b-15f63efc1d08" containerName="revoke-keys"
Jul 26 18:29:31.716937 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:31.716793    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd203e05-a7e9-4d9a-9667-c3c2338f9ee3" containerName="maas-api"
Jul 26 18:29:31.716937 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:31.716798    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd203e05-a7e9-4d9a-9667-c3c2338f9ee3" containerName="maas-api"
Jul 26 18:29:31.716937 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:31.716848    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f222359b-28b3-4fc9-872b-15f63efc1d08" containerName="revoke-keys"
Jul 26 18:29:31.716937 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:31.716861    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd203e05-a7e9-4d9a-9667-c3c2338f9ee3" containerName="maas-api"
Jul 26 18:29:31.721320 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:31.721301    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9"
Jul 26 18:29:31.724358 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:31.724339    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-57f93c-serving-cert\""
Jul 26 18:29:31.742876 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:31.742835    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9"]
Jul 26 18:29:31.860631 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:31.860598    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff60c926-d113-4a1a-ac00-faa9791b6cdd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9\" (UID: \"ff60c926-d113-4a1a-ac00-faa9791b6cdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9"
Jul 26 18:29:31.860819 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:31.860649    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58cxw\" (UniqueName: \"kubernetes.io/projected/ff60c926-d113-4a1a-ac00-faa9791b6cdd-kube-api-access-58cxw\") pod \"maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9\" (UID: \"ff60c926-d113-4a1a-ac00-faa9791b6cdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9"
Jul 26 18:29:31.860819 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:31.860717    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff60c926-d113-4a1a-ac00-faa9791b6cdd-maas-api-tls\") pod \"maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9\" (UID: \"ff60c926-d113-4a1a-ac00-faa9791b6cdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9"
Jul 26 18:29:31.961707 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:31.961671    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff60c926-d113-4a1a-ac00-faa9791b6cdd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9\" (UID: \"ff60c926-d113-4a1a-ac00-faa9791b6cdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9"
Jul 26 18:29:31.961892 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:31.961745    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58cxw\" (UniqueName: \"kubernetes.io/projected/ff60c926-d113-4a1a-ac00-faa9791b6cdd-kube-api-access-58cxw\") pod \"maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9\" (UID: \"ff60c926-d113-4a1a-ac00-faa9791b6cdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9"
Jul 26 18:29:31.961892 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:31.961773    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff60c926-d113-4a1a-ac00-faa9791b6cdd-maas-api-tls\") pod \"maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9\" (UID: \"ff60c926-d113-4a1a-ac00-faa9791b6cdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9"
Jul 26 18:29:31.961892 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:29:31.961866    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-serving-cert: secret "maas-api-e2e-shared-a-57f93c-serving-cert" not found
Jul 26 18:29:31.962029 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:29:31.961929    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff60c926-d113-4a1a-ac00-faa9791b6cdd-maas-api-tls podName:ff60c926-d113-4a1a-ac00-faa9791b6cdd nodeName:}" failed. No retries permitted until 2026-07-26 18:29:32.461911829 +0000 UTC m=+2429.114102671 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ff60c926-d113-4a1a-ac00-faa9791b6cdd-maas-api-tls") pod "maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9" (UID: "ff60c926-d113-4a1a-ac00-faa9791b6cdd") : secret "maas-api-e2e-shared-a-57f93c-serving-cert" not found
Jul 26 18:29:31.962506 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:31.962488    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff60c926-d113-4a1a-ac00-faa9791b6cdd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9\" (UID: \"ff60c926-d113-4a1a-ac00-faa9791b6cdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9"
Jul 26 18:29:31.970425 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:31.970363    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58cxw\" (UniqueName: \"kubernetes.io/projected/ff60c926-d113-4a1a-ac00-faa9791b6cdd-kube-api-access-58cxw\") pod \"maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9\" (UID: \"ff60c926-d113-4a1a-ac00-faa9791b6cdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9"
Jul 26 18:29:32.466135 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:32.466077    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff60c926-d113-4a1a-ac00-faa9791b6cdd-maas-api-tls\") pod \"maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9\" (UID: \"ff60c926-d113-4a1a-ac00-faa9791b6cdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9"
Jul 26 18:29:32.468780 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:32.468747    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff60c926-d113-4a1a-ac00-faa9791b6cdd-maas-api-tls\") pod \"maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9\" (UID: \"ff60c926-d113-4a1a-ac00-faa9791b6cdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9"
Jul 26 18:29:32.631362 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:32.631325    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9"
Jul 26 18:29:32.775586 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:32.775555    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9"]
Jul 26 18:29:32.777595 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:29:32.777568    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff60c926_d113_4a1a_ac00_faa9791b6cdd.slice/crio-c8d15cdcf96ebd3fcb8b61af56b2f62a730c2bede20774c7dd614c7d821cf07b WatchSource:0}: Error finding container c8d15cdcf96ebd3fcb8b61af56b2f62a730c2bede20774c7dd614c7d821cf07b: Status 404 returned error can't find the container with id c8d15cdcf96ebd3fcb8b61af56b2f62a730c2bede20774c7dd614c7d821cf07b
Jul 26 18:29:33.756807 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:33.756753    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9" event={"ID":"ff60c926-d113-4a1a-ac00-faa9791b6cdd","Type":"ContainerStarted","Data":"e8369c4fbb455a3e6abce790821ea9d3d1668210d7c57ea78161cc822586e57c"}
Jul 26 18:29:33.756993 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:33.756830    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9"
Jul 26 18:29:33.756993 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:33.756854    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9" event={"ID":"ff60c926-d113-4a1a-ac00-faa9791b6cdd","Type":"ContainerStarted","Data":"c8d15cdcf96ebd3fcb8b61af56b2f62a730c2bede20774c7dd614c7d821cf07b"}
Jul 26 18:29:33.784542 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:33.784486    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9" podStartSLOduration=2.396203226 podStartE2EDuration="2.784469505s" totalImagesPullingTime="388.266279ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:29:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:29:33.782752107 +0000 UTC m=+2430.434942965" watchObservedRunningTime="2026-07-26 18:29:33.784469505 +0000 UTC m=+2430.436660362"
Jul 26 18:29:39.736743 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:39.736709    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4"]
Jul 26 18:29:39.745863 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:39.745839    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4"
Jul 26 18:29:39.747976 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:39.747949    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-6aa894-serving-cert\""
Jul 26 18:29:39.751024 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:39.751000    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4"]
Jul 26 18:29:39.771091 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:39.771066    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9"
Jul 26 18:29:39.930977 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:39.930942    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39a258bd-2d71-45e2-b313-c4bbfc1800c6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4\" (UID: \"39a258bd-2d71-45e2-b313-c4bbfc1800c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4"
Jul 26 18:29:39.931177 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:39.931104    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39a258bd-2d71-45e2-b313-c4bbfc1800c6-maas-api-tls\") pod \"maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4\" (UID: \"39a258bd-2d71-45e2-b313-c4bbfc1800c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4"
Jul 26 18:29:39.931177 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:39.931162    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w24vd\" (UniqueName: \"kubernetes.io/projected/39a258bd-2d71-45e2-b313-c4bbfc1800c6-kube-api-access-w24vd\") pod \"maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4\" (UID: \"39a258bd-2d71-45e2-b313-c4bbfc1800c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4"
Jul 26 18:29:40.032507 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:40.032472    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39a258bd-2d71-45e2-b313-c4bbfc1800c6-maas-api-tls\") pod \"maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4\" (UID: \"39a258bd-2d71-45e2-b313-c4bbfc1800c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4"
Jul 26 18:29:40.032703 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:40.032528    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w24vd\" (UniqueName: \"kubernetes.io/projected/39a258bd-2d71-45e2-b313-c4bbfc1800c6-kube-api-access-w24vd\") pod \"maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4\" (UID: \"39a258bd-2d71-45e2-b313-c4bbfc1800c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4"
Jul 26 18:29:40.032703 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:40.032584    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39a258bd-2d71-45e2-b313-c4bbfc1800c6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4\" (UID: \"39a258bd-2d71-45e2-b313-c4bbfc1800c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4"
Jul 26 18:29:40.032703 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:29:40.032634    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-serving-cert: secret "maas-api-e2e-shared-b-6aa894-serving-cert" not found
Jul 26 18:29:40.032842 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:29:40.032713    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39a258bd-2d71-45e2-b313-c4bbfc1800c6-maas-api-tls podName:39a258bd-2d71-45e2-b313-c4bbfc1800c6 nodeName:}" failed. No retries permitted until 2026-07-26 18:29:40.532695414 +0000 UTC m=+2437.184886253 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/39a258bd-2d71-45e2-b313-c4bbfc1800c6-maas-api-tls") pod "maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4" (UID: "39a258bd-2d71-45e2-b313-c4bbfc1800c6") : secret "maas-api-e2e-shared-b-6aa894-serving-cert" not found
Jul 26 18:29:40.033207 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:40.033186    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39a258bd-2d71-45e2-b313-c4bbfc1800c6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4\" (UID: \"39a258bd-2d71-45e2-b313-c4bbfc1800c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4"
Jul 26 18:29:40.040965 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:40.040935    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w24vd\" (UniqueName: \"kubernetes.io/projected/39a258bd-2d71-45e2-b313-c4bbfc1800c6-kube-api-access-w24vd\") pod \"maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4\" (UID: \"39a258bd-2d71-45e2-b313-c4bbfc1800c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4"
Jul 26 18:29:40.536704 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:40.536661    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39a258bd-2d71-45e2-b313-c4bbfc1800c6-maas-api-tls\") pod \"maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4\" (UID: \"39a258bd-2d71-45e2-b313-c4bbfc1800c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4"
Jul 26 18:29:40.539125 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:40.539097    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39a258bd-2d71-45e2-b313-c4bbfc1800c6-maas-api-tls\") pod \"maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4\" (UID: \"39a258bd-2d71-45e2-b313-c4bbfc1800c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4"
Jul 26 18:29:40.658289 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:40.658249    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4"
Jul 26 18:29:40.797071 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:40.797048    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4"]
Jul 26 18:29:40.798738 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:29:40.798703    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39a258bd_2d71_45e2_b313_c4bbfc1800c6.slice/crio-d0ef616f365e7ed3f024b5813780000098c74e3822646ef279a62e85b89162ef WatchSource:0}: Error finding container d0ef616f365e7ed3f024b5813780000098c74e3822646ef279a62e85b89162ef: Status 404 returned error can't find the container with id d0ef616f365e7ed3f024b5813780000098c74e3822646ef279a62e85b89162ef
Jul 26 18:29:41.802916 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:41.802881    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4" event={"ID":"39a258bd-2d71-45e2-b313-c4bbfc1800c6","Type":"ContainerStarted","Data":"737de2582e54c3f258fb2d51f2ddc5c1e47876e213879689522725137745d1ac"}
Jul 26 18:29:41.803360 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:41.802960    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4" event={"ID":"39a258bd-2d71-45e2-b313-c4bbfc1800c6","Type":"ContainerStarted","Data":"d0ef616f365e7ed3f024b5813780000098c74e3822646ef279a62e85b89162ef"}
Jul 26 18:29:41.803360 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:41.803006    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4"
Jul 26 18:29:41.819146 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:41.819074    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4" podStartSLOduration=2.444632477 podStartE2EDuration="2.819058972s" totalImagesPullingTime="374.426495ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:29:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:29:41.818173813 +0000 UTC m=+2438.470364929" watchObservedRunningTime="2026-07-26 18:29:41.819058972 +0000 UTC m=+2438.471249830"
Jul 26 18:29:47.813666 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:29:47.813632    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4"
Jul 26 18:30:00.132371 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.132332    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6aa894-29751510-kwqw6"]
Jul 26 18:30:00.136167 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.136141    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751510-8p4tw"]
Jul 26 18:30:00.136328 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.136199    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6aa894-29751510-kwqw6"
Jul 26 18:30:00.138310 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.138291    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-swzk5\""
Jul 26 18:30:00.139502 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.139485    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-57f93c-29751510-rpcqg"]
Jul 26 18:30:00.139686 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.139659    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751510-8p4tw"
Jul 26 18:30:00.143270 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.143244    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6aa894-29751510-kwqw6"]
Jul 26 18:30:00.143395 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.143347    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-57f93c-29751510-rpcqg"
Jul 26 18:30:00.147065 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.147013    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751510-8p4tw"]
Jul 26 18:30:00.152902 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.152755    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6aa894-29751510-kwqw6"
Jul 26 18:30:00.163976 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.163940    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751510-8p4tw"
Jul 26 18:30:00.164263 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.164244    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-57f93c-29751510-rpcqg"]
Jul 26 18:30:00.198173 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.198144    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-57f93c-29751510-rpcqg"
Jul 26 18:30:00.321440 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.321127    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6aa894-29751510-kwqw6"]
Jul 26 18:30:00.323400 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:30:00.323360    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9522439_711f_4d22_af3f_790fa30cea7a.slice/crio-8e0c5ee291f2c32887e5e5c11676c81ed8e4d0beb950b777865d18daeafc9e17 WatchSource:0}: Error finding container 8e0c5ee291f2c32887e5e5c11676c81ed8e4d0beb950b777865d18daeafc9e17: Status 404 returned error can't find the container with id 8e0c5ee291f2c32887e5e5c11676c81ed8e4d0beb950b777865d18daeafc9e17
Jul 26 18:30:00.350466 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.350435    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751510-8p4tw"]
Jul 26 18:30:00.352393 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:30:00.352360    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc3f950f_3cf3_47ff_8aab_9709065ad09b.slice/crio-4beecf72c52d8275fd75c777b919845d91374b2d0f3eabab5a7e3f51d735369a WatchSource:0}: Error finding container 4beecf72c52d8275fd75c777b919845d91374b2d0f3eabab5a7e3f51d735369a: Status 404 returned error can't find the container with id 4beecf72c52d8275fd75c777b919845d91374b2d0f3eabab5a7e3f51d735369a
Jul 26 18:30:00.381596 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.381565    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-57f93c-29751510-rpcqg"]
Jul 26 18:30:00.388381 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:30:00.388349    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1509f49_c50f_4606_865d_08fc90687332.slice/crio-f7c0fc2caece0c65b9ff0138edddb8e89c8deb738576ab76bab4211fea6511d2 WatchSource:0}: Error finding container f7c0fc2caece0c65b9ff0138edddb8e89c8deb738576ab76bab4211fea6511d2: Status 404 returned error can't find the container with id f7c0fc2caece0c65b9ff0138edddb8e89c8deb738576ab76bab4211fea6511d2
Jul 26 18:30:00.907081 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.906963    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6aa894-29751510-kwqw6" event={"ID":"a9522439-711f-4d22-af3f-790fa30cea7a","Type":"ContainerStarted","Data":"423fb79affb977451eaace38a9364f755719b5fa510eeb47cb749e2114595a28"}
Jul 26 18:30:00.907081 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.907038    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6aa894-29751510-kwqw6" event={"ID":"a9522439-711f-4d22-af3f-790fa30cea7a","Type":"ContainerStarted","Data":"8e0c5ee291f2c32887e5e5c11676c81ed8e4d0beb950b777865d18daeafc9e17"}
Jul 26 18:30:00.908583 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.908559    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751510-8p4tw" event={"ID":"cc3f950f-3cf3-47ff-8aab-9709065ad09b","Type":"ContainerStarted","Data":"e2dc23346838c230b487e11f1ee2913f8ef7278853de6fa74aff6a3999c98067"}
Jul 26 18:30:00.908709 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.908600    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751510-8p4tw" event={"ID":"cc3f950f-3cf3-47ff-8aab-9709065ad09b","Type":"ContainerStarted","Data":"4beecf72c52d8275fd75c777b919845d91374b2d0f3eabab5a7e3f51d735369a"}
Jul 26 18:30:00.910013 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.909992    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-57f93c-29751510-rpcqg" event={"ID":"b1509f49-c50f-4606-865d-08fc90687332","Type":"ContainerStarted","Data":"edcd37ad483c92026d2fb50d5f840f41eb22ca227b48120299aab5bf2f099e5c"}
Jul 26 18:30:00.910136 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.910024    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-57f93c-29751510-rpcqg" event={"ID":"b1509f49-c50f-4606-865d-08fc90687332","Type":"ContainerStarted","Data":"f7c0fc2caece0c65b9ff0138edddb8e89c8deb738576ab76bab4211fea6511d2"}
Jul 26 18:30:00.925389 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.925333    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6aa894-29751510-kwqw6" podStartSLOduration=0.925318624 podStartE2EDuration="925.318624ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:30:00.922816953 +0000 UTC m=+2457.575007809" watchObservedRunningTime="2026-07-26 18:30:00.925318624 +0000 UTC m=+2457.577509484"
Jul 26 18:30:00.936570 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.936522    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751510-8p4tw" podStartSLOduration=0.936506363 podStartE2EDuration="936.506363ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:30:00.935882354 +0000 UTC m=+2457.588073212" watchObservedRunningTime="2026-07-26 18:30:00.936506363 +0000 UTC m=+2457.588697222"
Jul 26 18:30:00.951932 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:00.951879    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-57f93c-29751510-rpcqg" podStartSLOduration=0.951863049 podStartE2EDuration="951.863049ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:30:00.949967961 +0000 UTC m=+2457.602158831" watchObservedRunningTime="2026-07-26 18:30:00.951863049 +0000 UTC m=+2457.604053907"
Jul 26 18:30:01.917902 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:01.917863    2570 generic.go:356] "Generic (PLEG): container finished" podID="a9522439-711f-4d22-af3f-790fa30cea7a" containerID="423fb79affb977451eaace38a9364f755719b5fa510eeb47cb749e2114595a28" exitCode=0
Jul 26 18:30:01.918377 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:01.917949    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6aa894-29751510-kwqw6" event={"ID":"a9522439-711f-4d22-af3f-790fa30cea7a","Type":"ContainerDied","Data":"423fb79affb977451eaace38a9364f755719b5fa510eeb47cb749e2114595a28"}
Jul 26 18:30:01.919507 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:01.919484    2570 generic.go:356] "Generic (PLEG): container finished" podID="cc3f950f-3cf3-47ff-8aab-9709065ad09b" containerID="e2dc23346838c230b487e11f1ee2913f8ef7278853de6fa74aff6a3999c98067" exitCode=0
Jul 26 18:30:01.919619 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:01.919577    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751510-8p4tw" event={"ID":"cc3f950f-3cf3-47ff-8aab-9709065ad09b","Type":"ContainerDied","Data":"e2dc23346838c230b487e11f1ee2913f8ef7278853de6fa74aff6a3999c98067"}
Jul 26 18:30:01.920920 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:01.920898    2570 generic.go:356] "Generic (PLEG): container finished" podID="b1509f49-c50f-4606-865d-08fc90687332" containerID="edcd37ad483c92026d2fb50d5f840f41eb22ca227b48120299aab5bf2f099e5c" exitCode=0
Jul 26 18:30:01.921039 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:01.920976    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-57f93c-29751510-rpcqg" event={"ID":"b1509f49-c50f-4606-865d-08fc90687332","Type":"ContainerDied","Data":"edcd37ad483c92026d2fb50d5f840f41eb22ca227b48120299aab5bf2f099e5c"}
Jul 26 18:30:03.085849 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.085820    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751510-8p4tw"
Jul 26 18:30:03.089081 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.089051    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-57f93c-29751510-rpcqg"
Jul 26 18:30:03.103769 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.103740    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6aa894-29751510-kwqw6"
Jul 26 18:30:03.922460 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.922426    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"]
Jul 26 18:30:03.922783 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.922771    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc3f950f-3cf3-47ff-8aab-9709065ad09b" containerName="cleanup"
Jul 26 18:30:03.922827 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.922785    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc3f950f-3cf3-47ff-8aab-9709065ad09b" containerName="cleanup"
Jul 26 18:30:03.922827 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.922803    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9522439-711f-4d22-af3f-790fa30cea7a" containerName="cleanup"
Jul 26 18:30:03.922827 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.922808    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9522439-711f-4d22-af3f-790fa30cea7a" containerName="cleanup"
Jul 26 18:30:03.922827 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.922815    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1509f49-c50f-4606-865d-08fc90687332" containerName="cleanup"
Jul 26 18:30:03.922827 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.922821    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1509f49-c50f-4606-865d-08fc90687332" containerName="cleanup"
Jul 26 18:30:03.922982 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.922876    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1509f49-c50f-4606-865d-08fc90687332" containerName="cleanup"
Jul 26 18:30:03.922982 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.922884    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc3f950f-3cf3-47ff-8aab-9709065ad09b" containerName="cleanup"
Jul 26 18:30:03.922982 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.922891    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9522439-711f-4d22-af3f-790fa30cea7a" containerName="cleanup"
Jul 26 18:30:03.926024 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.926007    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:30:03.928531 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.928507    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-57f93c\"/\"openshift-service-ca.crt\""
Jul 26 18:30:03.929351 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.929330    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-57f93c\"/\"auth-test-model-57f93c-kserve-self-signed-certs\""
Jul 26 18:30:03.929454 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.929356    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-57f93c\"/\"kube-root-ca.crt\""
Jul 26 18:30:03.929454 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.929370    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-57f93c\"/\"default-dockercfg-4rlzp\""
Jul 26 18:30:03.935938 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.935918    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6aa894-29751510-kwqw6"
Jul 26 18:30:03.936047 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.935934    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6aa894-29751510-kwqw6" event={"ID":"a9522439-711f-4d22-af3f-790fa30cea7a","Type":"ContainerDied","Data":"8e0c5ee291f2c32887e5e5c11676c81ed8e4d0beb950b777865d18daeafc9e17"}
Jul 26 18:30:03.936047 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.935970    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e0c5ee291f2c32887e5e5c11676c81ed8e4d0beb950b777865d18daeafc9e17"
Jul 26 18:30:03.937270 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.937251    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751510-8p4tw"
Jul 26 18:30:03.937366 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.937277    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751510-8p4tw" event={"ID":"cc3f950f-3cf3-47ff-8aab-9709065ad09b","Type":"ContainerDied","Data":"4beecf72c52d8275fd75c777b919845d91374b2d0f3eabab5a7e3f51d735369a"}
Jul 26 18:30:03.937366 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.937317    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4beecf72c52d8275fd75c777b919845d91374b2d0f3eabab5a7e3f51d735369a"
Jul 26 18:30:03.938672 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.938654    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-57f93c-29751510-rpcqg"
Jul 26 18:30:03.938743 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.938671    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-57f93c-29751510-rpcqg" event={"ID":"b1509f49-c50f-4606-865d-08fc90687332","Type":"ContainerDied","Data":"f7c0fc2caece0c65b9ff0138edddb8e89c8deb738576ab76bab4211fea6511d2"}
Jul 26 18:30:03.938743 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.938707    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f7c0fc2caece0c65b9ff0138edddb8e89c8deb738576ab76bab4211fea6511d2"
Jul 26 18:30:03.943410 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:03.943386    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"]
Jul 26 18:30:04.017691 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:04.017657    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-dshm\") pod \"auth-test-model-57f93c-kserve-7fc4848f6c-v9brp\" (UID: \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\") " pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:30:04.017691 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:04.017691    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-tls-certs\") pod \"auth-test-model-57f93c-kserve-7fc4848f6c-v9brp\" (UID: \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\") " pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:30:04.017922 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:04.017714    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-home\") pod \"auth-test-model-57f93c-kserve-7fc4848f6c-v9brp\" (UID: \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\") " pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:30:04.017922 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:04.017737    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-tmp-dir\") pod \"auth-test-model-57f93c-kserve-7fc4848f6c-v9brp\" (UID: \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\") " pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:30:04.017922 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:04.017788    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-model-cache\") pod \"auth-test-model-57f93c-kserve-7fc4848f6c-v9brp\" (UID: \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\") " pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:30:04.017922 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:04.017853    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrkv5\" (UniqueName: \"kubernetes.io/projected/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-kube-api-access-nrkv5\") pod \"auth-test-model-57f93c-kserve-7fc4848f6c-v9brp\" (UID: \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\") " pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:30:04.118854 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:04.118739    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-model-cache\") pod \"auth-test-model-57f93c-kserve-7fc4848f6c-v9brp\" (UID: \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\") " pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:30:04.118854 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:04.118809    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrkv5\" (UniqueName: \"kubernetes.io/projected/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-kube-api-access-nrkv5\") pod \"auth-test-model-57f93c-kserve-7fc4848f6c-v9brp\" (UID: \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\") " pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:30:04.119392 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:04.118876    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-dshm\") pod \"auth-test-model-57f93c-kserve-7fc4848f6c-v9brp\" (UID: \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\") " pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:30:04.119392 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:04.118893    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-tls-certs\") pod \"auth-test-model-57f93c-kserve-7fc4848f6c-v9brp\" (UID: \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\") " pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:30:04.119392 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:04.118922    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-home\") pod \"auth-test-model-57f93c-kserve-7fc4848f6c-v9brp\" (UID: \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\") " pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:30:04.119392 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:04.118947    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-tmp-dir\") pod \"auth-test-model-57f93c-kserve-7fc4848f6c-v9brp\" (UID: \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\") " pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:30:04.119392 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:04.119226    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-model-cache\") pod \"auth-test-model-57f93c-kserve-7fc4848f6c-v9brp\" (UID: \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\") " pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:30:04.119392 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:04.119328    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-tmp-dir\") pod \"auth-test-model-57f93c-kserve-7fc4848f6c-v9brp\" (UID: \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\") " pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:30:04.119630 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:04.119399    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-home\") pod \"auth-test-model-57f93c-kserve-7fc4848f6c-v9brp\" (UID: \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\") " pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:30:04.121253 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:04.121232    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-dshm\") pod \"auth-test-model-57f93c-kserve-7fc4848f6c-v9brp\" (UID: \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\") " pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:30:04.121450 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:04.121434    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-tls-certs\") pod \"auth-test-model-57f93c-kserve-7fc4848f6c-v9brp\" (UID: \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\") " pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:30:04.128854 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:04.128828    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrkv5\" (UniqueName: \"kubernetes.io/projected/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-kube-api-access-nrkv5\") pod \"auth-test-model-57f93c-kserve-7fc4848f6c-v9brp\" (UID: \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\") " pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:30:04.235352 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:04.235310    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:30:04.392539 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:04.392511    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"]
Jul 26 18:30:04.394947 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:30:04.394922    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podffd8049a_3f6a_4c13_8bcd_4c25e113ae4e.slice/crio-7fb178f110e82729d7eba6dd43020218020bc39ce1d759ea341b472f1b931bc6 WatchSource:0}: Error finding container 7fb178f110e82729d7eba6dd43020218020bc39ce1d759ea341b472f1b931bc6: Status 404 returned error can't find the container with id 7fb178f110e82729d7eba6dd43020218020bc39ce1d759ea341b472f1b931bc6
Jul 26 18:30:04.947048 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:04.946956    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp" event={"ID":"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e","Type":"ContainerStarted","Data":"6594e07ee51d585fcef6d6788065cd1286236e16944588eca31ff1afaa7dabf1"}
Jul 26 18:30:04.947048 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:04.947000    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp" event={"ID":"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e","Type":"ContainerStarted","Data":"7fb178f110e82729d7eba6dd43020218020bc39ce1d759ea341b472f1b931bc6"}
Jul 26 18:30:04.980145 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:04.980066    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp" podStartSLOduration=1.980049539 podStartE2EDuration="1.980049539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:30:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:30:04.97795655 +0000 UTC m=+2461.630147408" watchObservedRunningTime="2026-07-26 18:30:04.980049539 +0000 UTC m=+2461.632240396"
Jul 26 18:30:14.235474 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:14.235432    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:30:14.235474 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:14.235487    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:30:14.248206 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:14.248172    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:30:15.014049 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:30:15.014016    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:34:04.064012 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:34:04.063932    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 18:34:04.073697 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:34:04.073674    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 18:34:04.312348 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:34:04.312315    2570 scope.go:117] "RemoveContainer" containerID="a00b92d057034534ddce69dd3966d75fe08e73fa423de1ae82bbbce70f26dc52"
Jul 26 18:35:04.336259 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:04.336225    2570 scope.go:117] "RemoveContainer" containerID="0f6bfbeb27f14e612067bca2a1290bad57ed92b5468c081a71bd698ad536c07e"
Jul 26 18:35:04.345145 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:04.345128    2570 scope.go:117] "RemoveContainer" containerID="e1a7aa0f11626bf7405bb45b1303892826d004fb2a4d7a66fabd148a96cc388b"
Jul 26 18:35:05.074375 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:05.074342    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"]
Jul 26 18:35:20.093551 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.093478    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp" podUID="ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e" containerName="main" containerID="cri-o://6594e07ee51d585fcef6d6788065cd1286236e16944588eca31ff1afaa7dabf1" gracePeriod=45
Jul 26 18:35:20.349594 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.349531    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:35:20.466163 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.466098    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-tmp-dir\") pod \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\" (UID: \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\") "
Jul 26 18:35:20.466365 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.466184    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-model-cache\") pod \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\" (UID: \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\") "
Jul 26 18:35:20.466365 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.466239    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrkv5\" (UniqueName: \"kubernetes.io/projected/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-kube-api-access-nrkv5\") pod \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\" (UID: \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\") "
Jul 26 18:35:20.466365 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.466286    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-tls-certs\") pod \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\" (UID: \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\") "
Jul 26 18:35:20.466365 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.466345    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-dshm\") pod \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\" (UID: \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\") "
Jul 26 18:35:20.466570 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.466386    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-home\") pod \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\" (UID: \"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e\") "
Jul 26 18:35:20.466570 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.466398    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e" (UID: "ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:35:20.466570 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.466507    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-model-cache" (OuterVolumeSpecName: "model-cache") pod "ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e" (UID: "ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:35:20.466689 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.466655    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-home" (OuterVolumeSpecName: "home") pod "ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e" (UID: "ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:35:20.466735 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.466717    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-tmp-dir\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:35:20.466780 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.466733    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-model-cache\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:35:20.468575 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.468544    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-dshm" (OuterVolumeSpecName: "dshm") pod "ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e" (UID: "ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:35:20.468689 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.468585    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e" (UID: "ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:35:20.468689 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.468672    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-kube-api-access-nrkv5" (OuterVolumeSpecName: "kube-api-access-nrkv5") pod "ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e" (UID: "ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e"). InnerVolumeSpecName "kube-api-access-nrkv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:35:20.568008 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.567964    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-home\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:35:20.568008 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.568002    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrkv5\" (UniqueName: \"kubernetes.io/projected/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-kube-api-access-nrkv5\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:35:20.568008 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.568016    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-tls-certs\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:35:20.568285 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.568031    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e-dshm\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:35:20.685036 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.684954    2570 generic.go:356] "Generic (PLEG): container finished" podID="ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e" containerID="6594e07ee51d585fcef6d6788065cd1286236e16944588eca31ff1afaa7dabf1" exitCode=0
Jul 26 18:35:20.685036 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.684998    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"
Jul 26 18:35:20.685261 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.685040    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp" event={"ID":"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e","Type":"ContainerDied","Data":"6594e07ee51d585fcef6d6788065cd1286236e16944588eca31ff1afaa7dabf1"}
Jul 26 18:35:20.685261 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.685087    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp" event={"ID":"ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e","Type":"ContainerDied","Data":"7fb178f110e82729d7eba6dd43020218020bc39ce1d759ea341b472f1b931bc6"}
Jul 26 18:35:20.685261 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.685133    2570 scope.go:117] "RemoveContainer" containerID="6594e07ee51d585fcef6d6788065cd1286236e16944588eca31ff1afaa7dabf1"
Jul 26 18:35:20.695052 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.695035    2570 scope.go:117] "RemoveContainer" containerID="6594e07ee51d585fcef6d6788065cd1286236e16944588eca31ff1afaa7dabf1"
Jul 26 18:35:20.695311 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:35:20.695293    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6594e07ee51d585fcef6d6788065cd1286236e16944588eca31ff1afaa7dabf1\": container with ID starting with 6594e07ee51d585fcef6d6788065cd1286236e16944588eca31ff1afaa7dabf1 not found: ID does not exist" containerID="6594e07ee51d585fcef6d6788065cd1286236e16944588eca31ff1afaa7dabf1"
Jul 26 18:35:20.695356 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.695319    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6594e07ee51d585fcef6d6788065cd1286236e16944588eca31ff1afaa7dabf1"} err="failed to get container status \"6594e07ee51d585fcef6d6788065cd1286236e16944588eca31ff1afaa7dabf1\": rpc error: code = NotFound desc = could not find container \"6594e07ee51d585fcef6d6788065cd1286236e16944588eca31ff1afaa7dabf1\": container with ID starting with 6594e07ee51d585fcef6d6788065cd1286236e16944588eca31ff1afaa7dabf1 not found: ID does not exist"
Jul 26 18:35:20.706703 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.706679    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"]
Jul 26 18:35:20.710742 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:20.710721    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-57f93c/auth-test-model-57f93c-kserve-7fc4848f6c-v9brp"]
Jul 26 18:35:21.838545 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:35:21.838508    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e" path="/var/lib/kubelet/pods/ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e/volumes"
Jul 26 18:36:17.581770 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.581730    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"]
Jul 26 18:36:17.582381 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.582360    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e" containerName="main"
Jul 26 18:36:17.582476 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.582383    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e" containerName="main"
Jul 26 18:36:17.583233 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.583206    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffd8049a-3f6a-4c13-8bcd-4c25e113ae4e" containerName="main"
Jul 26 18:36:17.589876 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.589847    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:36:17.593084 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.593053    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6aa894\"/\"default-dockercfg-qln6h\""
Jul 26 18:36:17.593228 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.593129    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6aa894\"/\"kube-root-ca.crt\""
Jul 26 18:36:17.593228 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.593145    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6aa894\"/\"openshift-service-ca.crt\""
Jul 26 18:36:17.593228 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.593167    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6aa894\"/\"sub-test-model-6aa894-kserve-self-signed-certs\""
Jul 26 18:36:17.598203 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.598178    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"]
Jul 26 18:36:17.769065 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.769030    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/724109cf-b6b1-49d9-bf00-91e63c807351-home\") pod \"sub-test-model-6aa894-kserve-557b5db9b9-rdcbw\" (UID: \"724109cf-b6b1-49d9-bf00-91e63c807351\") " pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:36:17.769065 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.769064    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/724109cf-b6b1-49d9-bf00-91e63c807351-model-cache\") pod \"sub-test-model-6aa894-kserve-557b5db9b9-rdcbw\" (UID: \"724109cf-b6b1-49d9-bf00-91e63c807351\") " pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:36:17.769291 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.769133    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/724109cf-b6b1-49d9-bf00-91e63c807351-tmp-dir\") pod \"sub-test-model-6aa894-kserve-557b5db9b9-rdcbw\" (UID: \"724109cf-b6b1-49d9-bf00-91e63c807351\") " pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:36:17.769291 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.769173    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/724109cf-b6b1-49d9-bf00-91e63c807351-dshm\") pod \"sub-test-model-6aa894-kserve-557b5db9b9-rdcbw\" (UID: \"724109cf-b6b1-49d9-bf00-91e63c807351\") " pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:36:17.769291 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.769191    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvnv2\" (UniqueName: \"kubernetes.io/projected/724109cf-b6b1-49d9-bf00-91e63c807351-kube-api-access-mvnv2\") pod \"sub-test-model-6aa894-kserve-557b5db9b9-rdcbw\" (UID: \"724109cf-b6b1-49d9-bf00-91e63c807351\") " pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:36:17.769291 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.769222    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/724109cf-b6b1-49d9-bf00-91e63c807351-tls-certs\") pod \"sub-test-model-6aa894-kserve-557b5db9b9-rdcbw\" (UID: \"724109cf-b6b1-49d9-bf00-91e63c807351\") " pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:36:17.869654 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.869575    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/724109cf-b6b1-49d9-bf00-91e63c807351-tls-certs\") pod \"sub-test-model-6aa894-kserve-557b5db9b9-rdcbw\" (UID: \"724109cf-b6b1-49d9-bf00-91e63c807351\") " pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:36:17.869654 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.869616    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/724109cf-b6b1-49d9-bf00-91e63c807351-home\") pod \"sub-test-model-6aa894-kserve-557b5db9b9-rdcbw\" (UID: \"724109cf-b6b1-49d9-bf00-91e63c807351\") " pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:36:17.869654 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.869632    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/724109cf-b6b1-49d9-bf00-91e63c807351-model-cache\") pod \"sub-test-model-6aa894-kserve-557b5db9b9-rdcbw\" (UID: \"724109cf-b6b1-49d9-bf00-91e63c807351\") " pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:36:17.869967 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.869703    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/724109cf-b6b1-49d9-bf00-91e63c807351-tmp-dir\") pod \"sub-test-model-6aa894-kserve-557b5db9b9-rdcbw\" (UID: \"724109cf-b6b1-49d9-bf00-91e63c807351\") " pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:36:17.869967 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.869740    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/724109cf-b6b1-49d9-bf00-91e63c807351-dshm\") pod \"sub-test-model-6aa894-kserve-557b5db9b9-rdcbw\" (UID: \"724109cf-b6b1-49d9-bf00-91e63c807351\") " pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:36:17.869967 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.869763    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvnv2\" (UniqueName: \"kubernetes.io/projected/724109cf-b6b1-49d9-bf00-91e63c807351-kube-api-access-mvnv2\") pod \"sub-test-model-6aa894-kserve-557b5db9b9-rdcbw\" (UID: \"724109cf-b6b1-49d9-bf00-91e63c807351\") " pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:36:17.870169 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.870037    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/724109cf-b6b1-49d9-bf00-91e63c807351-home\") pod \"sub-test-model-6aa894-kserve-557b5db9b9-rdcbw\" (UID: \"724109cf-b6b1-49d9-bf00-91e63c807351\") " pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:36:17.870169 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.870085    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/724109cf-b6b1-49d9-bf00-91e63c807351-tmp-dir\") pod \"sub-test-model-6aa894-kserve-557b5db9b9-rdcbw\" (UID: \"724109cf-b6b1-49d9-bf00-91e63c807351\") " pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:36:17.870169 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.870130    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/724109cf-b6b1-49d9-bf00-91e63c807351-model-cache\") pod \"sub-test-model-6aa894-kserve-557b5db9b9-rdcbw\" (UID: \"724109cf-b6b1-49d9-bf00-91e63c807351\") " pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:36:17.871949 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.871929    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/724109cf-b6b1-49d9-bf00-91e63c807351-dshm\") pod \"sub-test-model-6aa894-kserve-557b5db9b9-rdcbw\" (UID: \"724109cf-b6b1-49d9-bf00-91e63c807351\") " pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:36:17.872299 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.872282    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/724109cf-b6b1-49d9-bf00-91e63c807351-tls-certs\") pod \"sub-test-model-6aa894-kserve-557b5db9b9-rdcbw\" (UID: \"724109cf-b6b1-49d9-bf00-91e63c807351\") " pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:36:17.877360 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.877340    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvnv2\" (UniqueName: \"kubernetes.io/projected/724109cf-b6b1-49d9-bf00-91e63c807351-kube-api-access-mvnv2\") pod \"sub-test-model-6aa894-kserve-557b5db9b9-rdcbw\" (UID: \"724109cf-b6b1-49d9-bf00-91e63c807351\") " pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:36:17.901270 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:17.901247    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:36:18.033627 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:18.033596    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"]
Jul 26 18:36:18.035379 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:36:18.035349    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod724109cf_b6b1_49d9_bf00_91e63c807351.slice/crio-909f343f2f6c72074f0ff0df20a10c94c19ad7273f852545af31926a462069d1 WatchSource:0}: Error finding container 909f343f2f6c72074f0ff0df20a10c94c19ad7273f852545af31926a462069d1: Status 404 returned error can't find the container with id 909f343f2f6c72074f0ff0df20a10c94c19ad7273f852545af31926a462069d1
Jul 26 18:36:18.996983 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:18.996945    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw" event={"ID":"724109cf-b6b1-49d9-bf00-91e63c807351","Type":"ContainerStarted","Data":"095389e2ae8300737ceda745fb18b73281624b411fa2f29001660848d756506d"}
Jul 26 18:36:18.997409 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:18.996991    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw" event={"ID":"724109cf-b6b1-49d9-bf00-91e63c807351","Type":"ContainerStarted","Data":"909f343f2f6c72074f0ff0df20a10c94c19ad7273f852545af31926a462069d1"}
Jul 26 18:36:19.013759 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:19.013708    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw" podStartSLOduration=2.013692805 podStartE2EDuration="2.013692805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:36:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:36:19.012604003 +0000 UTC m=+2835.664794861" watchObservedRunningTime="2026-07-26 18:36:19.013692805 +0000 UTC m=+2835.665883663"
Jul 26 18:36:27.902293 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:27.902252    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:36:27.902744 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:27.902311    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:36:27.915148 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:27.915101    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:36:28.060226 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:36:28.060192    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:37:34.663614 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:34.663575    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"]
Jul 26 18:37:49.683660 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:49.683599    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw" podUID="724109cf-b6b1-49d9-bf00-91e63c807351" containerName="main" containerID="cri-o://095389e2ae8300737ceda745fb18b73281624b411fa2f29001660848d756506d" gracePeriod=45
Jul 26 18:37:49.939592 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:49.939530    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:37:50.026219 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.026190    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/724109cf-b6b1-49d9-bf00-91e63c807351-home\") pod \"724109cf-b6b1-49d9-bf00-91e63c807351\" (UID: \"724109cf-b6b1-49d9-bf00-91e63c807351\") "
Jul 26 18:37:50.026396 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.026242    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvnv2\" (UniqueName: \"kubernetes.io/projected/724109cf-b6b1-49d9-bf00-91e63c807351-kube-api-access-mvnv2\") pod \"724109cf-b6b1-49d9-bf00-91e63c807351\" (UID: \"724109cf-b6b1-49d9-bf00-91e63c807351\") "
Jul 26 18:37:50.026396 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.026294    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/724109cf-b6b1-49d9-bf00-91e63c807351-model-cache\") pod \"724109cf-b6b1-49d9-bf00-91e63c807351\" (UID: \"724109cf-b6b1-49d9-bf00-91e63c807351\") "
Jul 26 18:37:50.026396 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.026321    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/724109cf-b6b1-49d9-bf00-91e63c807351-dshm\") pod \"724109cf-b6b1-49d9-bf00-91e63c807351\" (UID: \"724109cf-b6b1-49d9-bf00-91e63c807351\") "
Jul 26 18:37:50.026396 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.026351    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/724109cf-b6b1-49d9-bf00-91e63c807351-tmp-dir\") pod \"724109cf-b6b1-49d9-bf00-91e63c807351\" (UID: \"724109cf-b6b1-49d9-bf00-91e63c807351\") "
Jul 26 18:37:50.026623 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.026419    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/724109cf-b6b1-49d9-bf00-91e63c807351-tls-certs\") pod \"724109cf-b6b1-49d9-bf00-91e63c807351\" (UID: \"724109cf-b6b1-49d9-bf00-91e63c807351\") "
Jul 26 18:37:50.026623 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.026518    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/724109cf-b6b1-49d9-bf00-91e63c807351-home" (OuterVolumeSpecName: "home") pod "724109cf-b6b1-49d9-bf00-91e63c807351" (UID: "724109cf-b6b1-49d9-bf00-91e63c807351"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:37:50.026623 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.026598    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/724109cf-b6b1-49d9-bf00-91e63c807351-model-cache" (OuterVolumeSpecName: "model-cache") pod "724109cf-b6b1-49d9-bf00-91e63c807351" (UID: "724109cf-b6b1-49d9-bf00-91e63c807351"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:37:50.026805 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.026641    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/724109cf-b6b1-49d9-bf00-91e63c807351-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "724109cf-b6b1-49d9-bf00-91e63c807351" (UID: "724109cf-b6b1-49d9-bf00-91e63c807351"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:37:50.026805 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.026691    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/724109cf-b6b1-49d9-bf00-91e63c807351-model-cache\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:37:50.026805 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.026707    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/724109cf-b6b1-49d9-bf00-91e63c807351-home\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:37:50.028670 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.028642    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/724109cf-b6b1-49d9-bf00-91e63c807351-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "724109cf-b6b1-49d9-bf00-91e63c807351" (UID: "724109cf-b6b1-49d9-bf00-91e63c807351"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:37:50.028784 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.028741    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/724109cf-b6b1-49d9-bf00-91e63c807351-kube-api-access-mvnv2" (OuterVolumeSpecName: "kube-api-access-mvnv2") pod "724109cf-b6b1-49d9-bf00-91e63c807351" (UID: "724109cf-b6b1-49d9-bf00-91e63c807351"). InnerVolumeSpecName "kube-api-access-mvnv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:37:50.028784 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.028744    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/724109cf-b6b1-49d9-bf00-91e63c807351-dshm" (OuterVolumeSpecName: "dshm") pod "724109cf-b6b1-49d9-bf00-91e63c807351" (UID: "724109cf-b6b1-49d9-bf00-91e63c807351"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:37:50.127376 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.127338    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/724109cf-b6b1-49d9-bf00-91e63c807351-tls-certs\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:37:50.127376 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.127370    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvnv2\" (UniqueName: \"kubernetes.io/projected/724109cf-b6b1-49d9-bf00-91e63c807351-kube-api-access-mvnv2\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:37:50.127376 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.127379    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/724109cf-b6b1-49d9-bf00-91e63c807351-dshm\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:37:50.127602 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.127411    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/724109cf-b6b1-49d9-bf00-91e63c807351-tmp-dir\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:37:50.503866 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.503829    2570 generic.go:356] "Generic (PLEG): container finished" podID="724109cf-b6b1-49d9-bf00-91e63c807351" containerID="095389e2ae8300737ceda745fb18b73281624b411fa2f29001660848d756506d" exitCode=0
Jul 26 18:37:50.504073 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.503883    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"
Jul 26 18:37:50.504073 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.503911    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw" event={"ID":"724109cf-b6b1-49d9-bf00-91e63c807351","Type":"ContainerDied","Data":"095389e2ae8300737ceda745fb18b73281624b411fa2f29001660848d756506d"}
Jul 26 18:37:50.504073 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.503961    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw" event={"ID":"724109cf-b6b1-49d9-bf00-91e63c807351","Type":"ContainerDied","Data":"909f343f2f6c72074f0ff0df20a10c94c19ad7273f852545af31926a462069d1"}
Jul 26 18:37:50.504073 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.503983    2570 scope.go:117] "RemoveContainer" containerID="095389e2ae8300737ceda745fb18b73281624b411fa2f29001660848d756506d"
Jul 26 18:37:50.513822 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.513804    2570 scope.go:117] "RemoveContainer" containerID="095389e2ae8300737ceda745fb18b73281624b411fa2f29001660848d756506d"
Jul 26 18:37:50.514074 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:37:50.514055    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"095389e2ae8300737ceda745fb18b73281624b411fa2f29001660848d756506d\": container with ID starting with 095389e2ae8300737ceda745fb18b73281624b411fa2f29001660848d756506d not found: ID does not exist" containerID="095389e2ae8300737ceda745fb18b73281624b411fa2f29001660848d756506d"
Jul 26 18:37:50.514154 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.514082    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"095389e2ae8300737ceda745fb18b73281624b411fa2f29001660848d756506d"} err="failed to get container status \"095389e2ae8300737ceda745fb18b73281624b411fa2f29001660848d756506d\": rpc error: code = NotFound desc = could not find container \"095389e2ae8300737ceda745fb18b73281624b411fa2f29001660848d756506d\": container with ID starting with 095389e2ae8300737ceda745fb18b73281624b411fa2f29001660848d756506d not found: ID does not exist"
Jul 26 18:37:50.524417 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.524396    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"]
Jul 26 18:37:50.530418 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:50.530396    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-6aa894/sub-test-model-6aa894-kserve-557b5db9b9-rdcbw"]
Jul 26 18:37:51.837685 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:37:51.837650    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="724109cf-b6b1-49d9-bf00-91e63c807351" path="/var/lib/kubelet/pods/724109cf-b6b1-49d9-bf00-91e63c807351/volumes"
Jul 26 18:38:12.295901 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.295869    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"]
Jul 26 18:38:12.296377 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.296245    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="724109cf-b6b1-49d9-bf00-91e63c807351" containerName="main"
Jul 26 18:38:12.296377 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.296256    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="724109cf-b6b1-49d9-bf00-91e63c807351" containerName="main"
Jul 26 18:38:12.296377 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.296329    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="724109cf-b6b1-49d9-bf00-91e63c807351" containerName="main"
Jul 26 18:38:12.300697 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.300680    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:38:12.303087 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.303068    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6aa894\"/\"openshift-service-ca.crt\""
Jul 26 18:38:12.303783 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.303766    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6aa894\"/\"rate-test-model-6aa894-kserve-self-signed-certs\""
Jul 26 18:38:12.304043 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.304015    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6aa894\"/\"kube-root-ca.crt\""
Jul 26 18:38:12.304170 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.304071    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6aa894\"/\"default-dockercfg-qln6h\""
Jul 26 18:38:12.310608 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.310588    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"]
Jul 26 18:38:12.427486 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.427454    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81d393f9-4c2f-46db-8456-02db554aab93-model-cache\") pod \"rate-test-model-6aa894-kserve-848bbdcccf-7q659\" (UID: \"81d393f9-4c2f-46db-8456-02db554aab93\") " pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:38:12.427486 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.427491    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81d393f9-4c2f-46db-8456-02db554aab93-tmp-dir\") pod \"rate-test-model-6aa894-kserve-848bbdcccf-7q659\" (UID: \"81d393f9-4c2f-46db-8456-02db554aab93\") " pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:38:12.427720 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.427518    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81d393f9-4c2f-46db-8456-02db554aab93-dshm\") pod \"rate-test-model-6aa894-kserve-848bbdcccf-7q659\" (UID: \"81d393f9-4c2f-46db-8456-02db554aab93\") " pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:38:12.427720 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.427560    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81d393f9-4c2f-46db-8456-02db554aab93-home\") pod \"rate-test-model-6aa894-kserve-848bbdcccf-7q659\" (UID: \"81d393f9-4c2f-46db-8456-02db554aab93\") " pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:38:12.427720 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.427596    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhfsh\" (UniqueName: \"kubernetes.io/projected/81d393f9-4c2f-46db-8456-02db554aab93-kube-api-access-hhfsh\") pod \"rate-test-model-6aa894-kserve-848bbdcccf-7q659\" (UID: \"81d393f9-4c2f-46db-8456-02db554aab93\") " pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:38:12.427720 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.427622    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81d393f9-4c2f-46db-8456-02db554aab93-tls-certs\") pod \"rate-test-model-6aa894-kserve-848bbdcccf-7q659\" (UID: \"81d393f9-4c2f-46db-8456-02db554aab93\") " pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:38:12.528713 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.528680    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81d393f9-4c2f-46db-8456-02db554aab93-model-cache\") pod \"rate-test-model-6aa894-kserve-848bbdcccf-7q659\" (UID: \"81d393f9-4c2f-46db-8456-02db554aab93\") " pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:38:12.528713 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.528715    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81d393f9-4c2f-46db-8456-02db554aab93-tmp-dir\") pod \"rate-test-model-6aa894-kserve-848bbdcccf-7q659\" (UID: \"81d393f9-4c2f-46db-8456-02db554aab93\") " pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:38:12.528922 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.528745    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81d393f9-4c2f-46db-8456-02db554aab93-dshm\") pod \"rate-test-model-6aa894-kserve-848bbdcccf-7q659\" (UID: \"81d393f9-4c2f-46db-8456-02db554aab93\") " pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:38:12.528922 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.528770    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81d393f9-4c2f-46db-8456-02db554aab93-home\") pod \"rate-test-model-6aa894-kserve-848bbdcccf-7q659\" (UID: \"81d393f9-4c2f-46db-8456-02db554aab93\") " pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:38:12.528922 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.528804    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhfsh\" (UniqueName: \"kubernetes.io/projected/81d393f9-4c2f-46db-8456-02db554aab93-kube-api-access-hhfsh\") pod \"rate-test-model-6aa894-kserve-848bbdcccf-7q659\" (UID: \"81d393f9-4c2f-46db-8456-02db554aab93\") " pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:38:12.528922 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.528844    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81d393f9-4c2f-46db-8456-02db554aab93-tls-certs\") pod \"rate-test-model-6aa894-kserve-848bbdcccf-7q659\" (UID: \"81d393f9-4c2f-46db-8456-02db554aab93\") " pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:38:12.529131 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.529063    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81d393f9-4c2f-46db-8456-02db554aab93-model-cache\") pod \"rate-test-model-6aa894-kserve-848bbdcccf-7q659\" (UID: \"81d393f9-4c2f-46db-8456-02db554aab93\") " pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:38:12.529219 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.529192    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81d393f9-4c2f-46db-8456-02db554aab93-tmp-dir\") pod \"rate-test-model-6aa894-kserve-848bbdcccf-7q659\" (UID: \"81d393f9-4c2f-46db-8456-02db554aab93\") " pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:38:12.529317 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.529300    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81d393f9-4c2f-46db-8456-02db554aab93-home\") pod \"rate-test-model-6aa894-kserve-848bbdcccf-7q659\" (UID: \"81d393f9-4c2f-46db-8456-02db554aab93\") " pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:38:12.531070 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.531050    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81d393f9-4c2f-46db-8456-02db554aab93-dshm\") pod \"rate-test-model-6aa894-kserve-848bbdcccf-7q659\" (UID: \"81d393f9-4c2f-46db-8456-02db554aab93\") " pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:38:12.531230 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.531213    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81d393f9-4c2f-46db-8456-02db554aab93-tls-certs\") pod \"rate-test-model-6aa894-kserve-848bbdcccf-7q659\" (UID: \"81d393f9-4c2f-46db-8456-02db554aab93\") " pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:38:12.536338 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.536316    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhfsh\" (UniqueName: \"kubernetes.io/projected/81d393f9-4c2f-46db-8456-02db554aab93-kube-api-access-hhfsh\") pod \"rate-test-model-6aa894-kserve-848bbdcccf-7q659\" (UID: \"81d393f9-4c2f-46db-8456-02db554aab93\") " pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:38:12.610532 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.610457    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:38:12.743464 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:12.743439    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"]
Jul 26 18:38:12.744945 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:38:12.744922    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81d393f9_4c2f_46db_8456_02db554aab93.slice/crio-6e7805cde9eb5acde49fac8a3e5c4039196ec3108f8ace7db7177af9fed2c096 WatchSource:0}: Error finding container 6e7805cde9eb5acde49fac8a3e5c4039196ec3108f8ace7db7177af9fed2c096: Status 404 returned error can't find the container with id 6e7805cde9eb5acde49fac8a3e5c4039196ec3108f8ace7db7177af9fed2c096
Jul 26 18:38:13.631556 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:13.631519    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659" event={"ID":"81d393f9-4c2f-46db-8456-02db554aab93","Type":"ContainerStarted","Data":"fcb1589ac1d9091041187887ed0c9ab7b1edbce0585f75695f3f7d22a44d9852"}
Jul 26 18:38:13.632023 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:13.631567    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659" event={"ID":"81d393f9-4c2f-46db-8456-02db554aab93","Type":"ContainerStarted","Data":"6e7805cde9eb5acde49fac8a3e5c4039196ec3108f8ace7db7177af9fed2c096"}
Jul 26 18:38:13.648067 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:13.648018    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659" podStartSLOduration=1.648003318 podStartE2EDuration="1.648003318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:38:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:38:13.646805427 +0000 UTC m=+2950.298996285" watchObservedRunningTime="2026-07-26 18:38:13.648003318 +0000 UTC m=+2950.300194176"
Jul 26 18:38:22.611575 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:22.611534    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:38:22.612013 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:22.611593    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:38:22.624375 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:22.624352    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:38:22.690239 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:38:22.690205    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:39:04.098706 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:04.098611    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 18:39:04.110144 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:04.110108    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 18:39:26.985697 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:26.985659    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"]
Jul 26 18:39:27.921004 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:27.920969    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7795-140293211299120"]
Jul 26 18:39:27.924530 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:27.924512    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7795-140293211299120"
Jul 26 18:39:27.926625 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:27.926601    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7rr9f\""
Jul 26 18:39:27.931879 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:27.931858    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7795-140293211299120"]
Jul 26 18:39:27.986918 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:27.986885    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz2bp\" (UniqueName: \"kubernetes.io/projected/83c17750-de73-4af5-bff3-dabdc2f51aa5-kube-api-access-vz2bp\") pod \"test-curl-7795-140293211299120\" (UID: \"83c17750-de73-4af5-bff3-dabdc2f51aa5\") " pod="opendatahub/test-curl-7795-140293211299120"
Jul 26 18:39:28.088273 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:28.088237    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz2bp\" (UniqueName: \"kubernetes.io/projected/83c17750-de73-4af5-bff3-dabdc2f51aa5-kube-api-access-vz2bp\") pod \"test-curl-7795-140293211299120\" (UID: \"83c17750-de73-4af5-bff3-dabdc2f51aa5\") " pod="opendatahub/test-curl-7795-140293211299120"
Jul 26 18:39:28.096808 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:28.096778    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz2bp\" (UniqueName: \"kubernetes.io/projected/83c17750-de73-4af5-bff3-dabdc2f51aa5-kube-api-access-vz2bp\") pod \"test-curl-7795-140293211299120\" (UID: \"83c17750-de73-4af5-bff3-dabdc2f51aa5\") " pod="opendatahub/test-curl-7795-140293211299120"
Jul 26 18:39:28.236837 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:28.236756    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7795-140293211299120"
Jul 26 18:39:28.368222 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:28.368194    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7795-140293211299120"]
Jul 26 18:39:28.370095 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:39:28.370068    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83c17750_de73_4af5_bff3_dabdc2f51aa5.slice/crio-d3ba990cee665d1dce63861a9cb9e6de9872e43af0ba7b7bec134266d14b6749 WatchSource:0}: Error finding container d3ba990cee665d1dce63861a9cb9e6de9872e43af0ba7b7bec134266d14b6749: Status 404 returned error can't find the container with id d3ba990cee665d1dce63861a9cb9e6de9872e43af0ba7b7bec134266d14b6749
Jul 26 18:39:28.371444 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:28.371420    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 18:39:29.035394 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:29.035358    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7795-140293211299120" event={"ID":"83c17750-de73-4af5-bff3-dabdc2f51aa5","Type":"ContainerStarted","Data":"d3ba990cee665d1dce63861a9cb9e6de9872e43af0ba7b7bec134266d14b6749"}
Jul 26 18:39:30.042768 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:30.042733    2570 generic.go:356] "Generic (PLEG): container finished" podID="83c17750-de73-4af5-bff3-dabdc2f51aa5" containerID="fb9bd5255544763bfc4c120218bbcbecc6a3049becff3aa7e183c935cadb4002" exitCode=0
Jul 26 18:39:30.043186 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:30.042820    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7795-140293211299120" event={"ID":"83c17750-de73-4af5-bff3-dabdc2f51aa5","Type":"ContainerDied","Data":"fb9bd5255544763bfc4c120218bbcbecc6a3049becff3aa7e183c935cadb4002"}
Jul 26 18:39:31.181822 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:31.181798    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7795-140293211299120"
Jul 26 18:39:31.221102 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:31.221074    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7795-140293211299120_83c17750-de73-4af5-bff3-dabdc2f51aa5/test-curl-7795-140293211299120/0.log"
Jul 26 18:39:31.247660 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:31.247629    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7795-140293211299120"]
Jul 26 18:39:31.254250 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:31.254222    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7795-140293211299120"]
Jul 26 18:39:31.321738 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:31.321678    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vz2bp\" (UniqueName: \"kubernetes.io/projected/83c17750-de73-4af5-bff3-dabdc2f51aa5-kube-api-access-vz2bp\") pod \"83c17750-de73-4af5-bff3-dabdc2f51aa5\" (UID: \"83c17750-de73-4af5-bff3-dabdc2f51aa5\") "
Jul 26 18:39:31.323974 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:31.323942    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83c17750-de73-4af5-bff3-dabdc2f51aa5-kube-api-access-vz2bp" (OuterVolumeSpecName: "kube-api-access-vz2bp") pod "83c17750-de73-4af5-bff3-dabdc2f51aa5" (UID: "83c17750-de73-4af5-bff3-dabdc2f51aa5"). InnerVolumeSpecName "kube-api-access-vz2bp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:39:31.380353 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:31.380316    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7795-140293211296176"]
Jul 26 18:39:31.380698 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:31.380685    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83c17750-de73-4af5-bff3-dabdc2f51aa5" containerName="test-curl-7795-140293211299120"
Jul 26 18:39:31.380741 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:31.380700    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="83c17750-de73-4af5-bff3-dabdc2f51aa5" containerName="test-curl-7795-140293211299120"
Jul 26 18:39:31.380781 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:31.380773    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="83c17750-de73-4af5-bff3-dabdc2f51aa5" containerName="test-curl-7795-140293211299120"
Jul 26 18:39:31.383760 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:31.383745    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7795-140293211296176"
Jul 26 18:39:31.390817 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:31.390795    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7795-140293211296176"]
Jul 26 18:39:31.422726 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:31.422682    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vz2bp\" (UniqueName: \"kubernetes.io/projected/83c17750-de73-4af5-bff3-dabdc2f51aa5-kube-api-access-vz2bp\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:39:31.524014 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:31.523979    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9954\" (UniqueName: \"kubernetes.io/projected/e8b774b3-f7da-4ef6-9ea4-c2706ad116f3-kube-api-access-b9954\") pod \"test-curl-7795-140293211296176\" (UID: \"e8b774b3-f7da-4ef6-9ea4-c2706ad116f3\") " pod="opendatahub/test-curl-7795-140293211296176"
Jul 26 18:39:31.625178 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:31.625058    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9954\" (UniqueName: \"kubernetes.io/projected/e8b774b3-f7da-4ef6-9ea4-c2706ad116f3-kube-api-access-b9954\") pod \"test-curl-7795-140293211296176\" (UID: \"e8b774b3-f7da-4ef6-9ea4-c2706ad116f3\") " pod="opendatahub/test-curl-7795-140293211296176"
Jul 26 18:39:31.633896 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:31.633861    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9954\" (UniqueName: \"kubernetes.io/projected/e8b774b3-f7da-4ef6-9ea4-c2706ad116f3-kube-api-access-b9954\") pod \"test-curl-7795-140293211296176\" (UID: \"e8b774b3-f7da-4ef6-9ea4-c2706ad116f3\") " pod="opendatahub/test-curl-7795-140293211296176"
Jul 26 18:39:31.694423 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:31.694382    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7795-140293211296176"
Jul 26 18:39:31.827455 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:31.827422    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7795-140293211296176"]
Jul 26 18:39:31.828482 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:39:31.828458    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8b774b3_f7da_4ef6_9ea4_c2706ad116f3.slice/crio-4fd20b6318a34a24c828070528f5959b64130054420be374962df85c7d97f0a9 WatchSource:0}: Error finding container 4fd20b6318a34a24c828070528f5959b64130054420be374962df85c7d97f0a9: Status 404 returned error can't find the container with id 4fd20b6318a34a24c828070528f5959b64130054420be374962df85c7d97f0a9
Jul 26 18:39:31.838714 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:31.838686    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83c17750-de73-4af5-bff3-dabdc2f51aa5" path="/var/lib/kubelet/pods/83c17750-de73-4af5-bff3-dabdc2f51aa5/volumes"
Jul 26 18:39:32.055349 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:32.055316    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7795-140293211296176" event={"ID":"e8b774b3-f7da-4ef6-9ea4-c2706ad116f3","Type":"ContainerStarted","Data":"4fd20b6318a34a24c828070528f5959b64130054420be374962df85c7d97f0a9"}
Jul 26 18:39:32.056779 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:32.056744    2570 scope.go:117] "RemoveContainer" containerID="fb9bd5255544763bfc4c120218bbcbecc6a3049becff3aa7e183c935cadb4002"
Jul 26 18:39:32.056920 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:32.056750    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7795-140293211299120"
Jul 26 18:39:33.064616 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:33.064582    2570 generic.go:356] "Generic (PLEG): container finished" podID="e8b774b3-f7da-4ef6-9ea4-c2706ad116f3" containerID="f46f4583a7884532f28562796722cfbec77db38b14657ac4d93441c19a755a0b" exitCode=0
Jul 26 18:39:33.065042 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:33.064667    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7795-140293211296176" event={"ID":"e8b774b3-f7da-4ef6-9ea4-c2706ad116f3","Type":"ContainerDied","Data":"f46f4583a7884532f28562796722cfbec77db38b14657ac4d93441c19a755a0b"}
Jul 26 18:39:34.204777 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:34.204748    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7795-140293211296176"
Jul 26 18:39:34.224149 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:34.224107    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7795-140293211296176_e8b774b3-f7da-4ef6-9ea4-c2706ad116f3/test-curl-7795-140293211296176/0.log"
Jul 26 18:39:34.248702 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:34.248671    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7795-140293211296176"]
Jul 26 18:39:34.254107 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:34.254080    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7795-140293211296176"]
Jul 26 18:39:34.349123 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:34.349041    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9954\" (UniqueName: \"kubernetes.io/projected/e8b774b3-f7da-4ef6-9ea4-c2706ad116f3-kube-api-access-b9954\") pod \"e8b774b3-f7da-4ef6-9ea4-c2706ad116f3\" (UID: \"e8b774b3-f7da-4ef6-9ea4-c2706ad116f3\") "
Jul 26 18:39:34.351241 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:34.351221    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8b774b3-f7da-4ef6-9ea4-c2706ad116f3-kube-api-access-b9954" (OuterVolumeSpecName: "kube-api-access-b9954") pod "e8b774b3-f7da-4ef6-9ea4-c2706ad116f3" (UID: "e8b774b3-f7da-4ef6-9ea4-c2706ad116f3"). InnerVolumeSpecName "kube-api-access-b9954". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:39:34.418899 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:34.418871    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7795-140293206599552"]
Jul 26 18:39:34.419280 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:34.419267    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8b774b3-f7da-4ef6-9ea4-c2706ad116f3" containerName="test-curl-7795-140293211296176"
Jul 26 18:39:34.419338 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:34.419282    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8b774b3-f7da-4ef6-9ea4-c2706ad116f3" containerName="test-curl-7795-140293211296176"
Jul 26 18:39:34.419375 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:34.419348    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8b774b3-f7da-4ef6-9ea4-c2706ad116f3" containerName="test-curl-7795-140293211296176"
Jul 26 18:39:34.422532 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:34.422517    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7795-140293206599552"
Jul 26 18:39:34.429225 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:34.429201    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7795-140293206599552"]
Jul 26 18:39:34.450333 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:34.450307    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9954\" (UniqueName: \"kubernetes.io/projected/e8b774b3-f7da-4ef6-9ea4-c2706ad116f3-kube-api-access-b9954\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:39:34.551145 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:34.551067    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qhs5\" (UniqueName: \"kubernetes.io/projected/6aace767-82e7-47fb-b339-1adccc904e1f-kube-api-access-2qhs5\") pod \"test-curl-7795-140293206599552\" (UID: \"6aace767-82e7-47fb-b339-1adccc904e1f\") " pod="opendatahub/test-curl-7795-140293206599552"
Jul 26 18:39:34.652583 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:34.652489    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qhs5\" (UniqueName: \"kubernetes.io/projected/6aace767-82e7-47fb-b339-1adccc904e1f-kube-api-access-2qhs5\") pod \"test-curl-7795-140293206599552\" (UID: \"6aace767-82e7-47fb-b339-1adccc904e1f\") " pod="opendatahub/test-curl-7795-140293206599552"
Jul 26 18:39:34.660727 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:34.660701    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qhs5\" (UniqueName: \"kubernetes.io/projected/6aace767-82e7-47fb-b339-1adccc904e1f-kube-api-access-2qhs5\") pod \"test-curl-7795-140293206599552\" (UID: \"6aace767-82e7-47fb-b339-1adccc904e1f\") " pod="opendatahub/test-curl-7795-140293206599552"
Jul 26 18:39:34.733937 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:34.733910    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7795-140293206599552"
Jul 26 18:39:34.868718 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:34.868692    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7795-140293206599552"]
Jul 26 18:39:34.869948 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:39:34.869917    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6aace767_82e7_47fb_b339_1adccc904e1f.slice/crio-25c7bb47006c1c55005924f5114fec6e1b1ba1fe39fb82358e7e7b7e81e94910 WatchSource:0}: Error finding container 25c7bb47006c1c55005924f5114fec6e1b1ba1fe39fb82358e7e7b7e81e94910: Status 404 returned error can't find the container with id 25c7bb47006c1c55005924f5114fec6e1b1ba1fe39fb82358e7e7b7e81e94910
Jul 26 18:39:35.076669 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:35.076622    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7795-140293206599552" event={"ID":"6aace767-82e7-47fb-b339-1adccc904e1f","Type":"ContainerStarted","Data":"25c7bb47006c1c55005924f5114fec6e1b1ba1fe39fb82358e7e7b7e81e94910"}
Jul 26 18:39:35.077873 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:35.077853    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4fd20b6318a34a24c828070528f5959b64130054420be374962df85c7d97f0a9"
Jul 26 18:39:35.077986 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:35.077887    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7795-140293211296176"
Jul 26 18:39:35.088461 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:39:35.088437    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7795-140293211296176\" is forbidden: User \"system:node:ip-10-0-128-45.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-45.ec2.internal' and this object" podUID="e8b774b3-f7da-4ef6-9ea4-c2706ad116f3" pod="opendatahub/test-curl-7795-140293211296176"
Jul 26 18:39:35.838144 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:35.838081    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8b774b3-f7da-4ef6-9ea4-c2706ad116f3" path="/var/lib/kubelet/pods/e8b774b3-f7da-4ef6-9ea4-c2706ad116f3/volumes"
Jul 26 18:39:36.084594 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:36.084561    2570 generic.go:356] "Generic (PLEG): container finished" podID="6aace767-82e7-47fb-b339-1adccc904e1f" containerID="1bdea773a2b3b0c8cfaa441f4f0328e2aca50780f499de153109d74a866e78e5" exitCode=0
Jul 26 18:39:36.084808 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:36.084650    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7795-140293206599552" event={"ID":"6aace767-82e7-47fb-b339-1adccc904e1f","Type":"ContainerDied","Data":"1bdea773a2b3b0c8cfaa441f4f0328e2aca50780f499de153109d74a866e78e5"}
Jul 26 18:39:37.224873 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:37.224846    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7795-140293206599552"
Jul 26 18:39:37.243625 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:37.243593    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7795-140293206599552_6aace767-82e7-47fb-b339-1adccc904e1f/test-curl-7795-140293206599552/0.log"
Jul 26 18:39:37.269256 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:37.269231    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7795-140293206599552"]
Jul 26 18:39:37.271486 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:37.271464    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7795-140293206599552"]
Jul 26 18:39:37.379377 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:37.379303    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qhs5\" (UniqueName: \"kubernetes.io/projected/6aace767-82e7-47fb-b339-1adccc904e1f-kube-api-access-2qhs5\") pod \"6aace767-82e7-47fb-b339-1adccc904e1f\" (UID: \"6aace767-82e7-47fb-b339-1adccc904e1f\") "
Jul 26 18:39:37.381427 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:37.381396    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6aace767-82e7-47fb-b339-1adccc904e1f-kube-api-access-2qhs5" (OuterVolumeSpecName: "kube-api-access-2qhs5") pod "6aace767-82e7-47fb-b339-1adccc904e1f" (UID: "6aace767-82e7-47fb-b339-1adccc904e1f"). InnerVolumeSpecName "kube-api-access-2qhs5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:39:37.388404 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:37.388384    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7795-140293206599552"]
Jul 26 18:39:37.388746 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:37.388734    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6aace767-82e7-47fb-b339-1adccc904e1f" containerName="test-curl-7795-140293206599552"
Jul 26 18:39:37.388792 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:37.388748    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6aace767-82e7-47fb-b339-1adccc904e1f" containerName="test-curl-7795-140293206599552"
Jul 26 18:39:37.388851 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:37.388841    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="6aace767-82e7-47fb-b339-1adccc904e1f" containerName="test-curl-7795-140293206599552"
Jul 26 18:39:37.390590 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:37.390576    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7795-140293206599552"
Jul 26 18:39:37.398919 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:37.398897    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7795-140293206599552"]
Jul 26 18:39:37.480080 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:37.480050    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmzsv\" (UniqueName: \"kubernetes.io/projected/b17b0d5c-1510-4c5b-8b1d-86e64b423ae8-kube-api-access-qmzsv\") pod \"test-curl-7795-140293206599552\" (UID: \"b17b0d5c-1510-4c5b-8b1d-86e64b423ae8\") " pod="opendatahub/test-curl-7795-140293206599552"
Jul 26 18:39:37.480258 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:37.480101    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qhs5\" (UniqueName: \"kubernetes.io/projected/6aace767-82e7-47fb-b339-1adccc904e1f-kube-api-access-2qhs5\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:39:37.581398 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:37.581367    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmzsv\" (UniqueName: \"kubernetes.io/projected/b17b0d5c-1510-4c5b-8b1d-86e64b423ae8-kube-api-access-qmzsv\") pod \"test-curl-7795-140293206599552\" (UID: \"b17b0d5c-1510-4c5b-8b1d-86e64b423ae8\") " pod="opendatahub/test-curl-7795-140293206599552"
Jul 26 18:39:37.589731 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:37.589712    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmzsv\" (UniqueName: \"kubernetes.io/projected/b17b0d5c-1510-4c5b-8b1d-86e64b423ae8-kube-api-access-qmzsv\") pod \"test-curl-7795-140293206599552\" (UID: \"b17b0d5c-1510-4c5b-8b1d-86e64b423ae8\") " pod="opendatahub/test-curl-7795-140293206599552"
Jul 26 18:39:37.700887 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:37.700805    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7795-140293206599552"
Jul 26 18:39:37.830972 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:37.830933    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7795-140293206599552"]
Jul 26 18:39:37.832289 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:39:37.832261    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb17b0d5c_1510_4c5b_8b1d_86e64b423ae8.slice/crio-0a1024f0e83b12df67ec8e88422cb99dc6cf7466b7074aa6ae9a24424ff3371f WatchSource:0}: Error finding container 0a1024f0e83b12df67ec8e88422cb99dc6cf7466b7074aa6ae9a24424ff3371f: Status 404 returned error can't find the container with id 0a1024f0e83b12df67ec8e88422cb99dc6cf7466b7074aa6ae9a24424ff3371f
Jul 26 18:39:37.838442 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:37.838422    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6aace767-82e7-47fb-b339-1adccc904e1f" path="/var/lib/kubelet/pods/6aace767-82e7-47fb-b339-1adccc904e1f/volumes"
Jul 26 18:39:38.097143 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:38.097091    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7795-140293206599552" event={"ID":"b17b0d5c-1510-4c5b-8b1d-86e64b423ae8","Type":"ContainerStarted","Data":"0a1024f0e83b12df67ec8e88422cb99dc6cf7466b7074aa6ae9a24424ff3371f"}
Jul 26 18:39:38.098449 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:38.098431    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7795-140293206599552"
Jul 26 18:39:38.098449 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:38.098442    2570 scope.go:117] "RemoveContainer" containerID="1bdea773a2b3b0c8cfaa441f4f0328e2aca50780f499de153109d74a866e78e5"
Jul 26 18:39:39.105584 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:39.105547    2570 generic.go:356] "Generic (PLEG): container finished" podID="b17b0d5c-1510-4c5b-8b1d-86e64b423ae8" containerID="0894d6603399e2dd5fbd7cd98a80419935e5a5af7570ce85b79eecd4d48d2ad2" exitCode=0
Jul 26 18:39:39.105997 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:39.105638    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7795-140293206599552" event={"ID":"b17b0d5c-1510-4c5b-8b1d-86e64b423ae8","Type":"ContainerDied","Data":"0894d6603399e2dd5fbd7cd98a80419935e5a5af7570ce85b79eecd4d48d2ad2"}
Jul 26 18:39:40.267505 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:40.267474    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7795-140293206599552"
Jul 26 18:39:40.287100 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:40.287072    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7795-140293206599552_b17b0d5c-1510-4c5b-8b1d-86e64b423ae8/test-curl-7795-140293206599552/0.log"
Jul 26 18:39:40.310099 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:40.310070    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7795-140293206599552"]
Jul 26 18:39:40.313962 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:40.313941    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7795-140293206599552"]
Jul 26 18:39:40.408892 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:40.408820    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmzsv\" (UniqueName: \"kubernetes.io/projected/b17b0d5c-1510-4c5b-8b1d-86e64b423ae8-kube-api-access-qmzsv\") pod \"b17b0d5c-1510-4c5b-8b1d-86e64b423ae8\" (UID: \"b17b0d5c-1510-4c5b-8b1d-86e64b423ae8\") "
Jul 26 18:39:40.410863 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:40.410835    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b17b0d5c-1510-4c5b-8b1d-86e64b423ae8-kube-api-access-qmzsv" (OuterVolumeSpecName: "kube-api-access-qmzsv") pod "b17b0d5c-1510-4c5b-8b1d-86e64b423ae8" (UID: "b17b0d5c-1510-4c5b-8b1d-86e64b423ae8"). InnerVolumeSpecName "kube-api-access-qmzsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:39:40.434511 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:40.434484    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7795-140293211309664"]
Jul 26 18:39:40.434863 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:40.434850    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b17b0d5c-1510-4c5b-8b1d-86e64b423ae8" containerName="test-curl-7795-140293206599552"
Jul 26 18:39:40.434905 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:40.434865    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b17b0d5c-1510-4c5b-8b1d-86e64b423ae8" containerName="test-curl-7795-140293206599552"
Jul 26 18:39:40.434940 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:40.434920    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b17b0d5c-1510-4c5b-8b1d-86e64b423ae8" containerName="test-curl-7795-140293206599552"
Jul 26 18:39:40.436758 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:40.436743    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7795-140293211309664"
Jul 26 18:39:40.443080 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:40.443050    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7795-140293211309664"]
Jul 26 18:39:40.510422 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:40.510389    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnfbd\" (UniqueName: \"kubernetes.io/projected/37b26d07-144d-4d7e-ba8a-d4327b760ab2-kube-api-access-cnfbd\") pod \"test-curl-7795-140293211309664\" (UID: \"37b26d07-144d-4d7e-ba8a-d4327b760ab2\") " pod="opendatahub/test-curl-7795-140293211309664"
Jul 26 18:39:40.510587 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:40.510442    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmzsv\" (UniqueName: \"kubernetes.io/projected/b17b0d5c-1510-4c5b-8b1d-86e64b423ae8-kube-api-access-qmzsv\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:39:40.611497 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:40.611465    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnfbd\" (UniqueName: \"kubernetes.io/projected/37b26d07-144d-4d7e-ba8a-d4327b760ab2-kube-api-access-cnfbd\") pod \"test-curl-7795-140293211309664\" (UID: \"37b26d07-144d-4d7e-ba8a-d4327b760ab2\") " pod="opendatahub/test-curl-7795-140293211309664"
Jul 26 18:39:40.618999 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:40.618969    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnfbd\" (UniqueName: \"kubernetes.io/projected/37b26d07-144d-4d7e-ba8a-d4327b760ab2-kube-api-access-cnfbd\") pod \"test-curl-7795-140293211309664\" (UID: \"37b26d07-144d-4d7e-ba8a-d4327b760ab2\") " pod="opendatahub/test-curl-7795-140293211309664"
Jul 26 18:39:40.747500 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:40.747393    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7795-140293211309664"
Jul 26 18:39:40.897428 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:40.897399    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7795-140293211309664"]
Jul 26 18:39:40.899055 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:39:40.899028    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod37b26d07_144d_4d7e_ba8a_d4327b760ab2.slice/crio-ce243599dd3f8161745c8962ec143bdccfae207b3988778c8a5a521cf1eca551 WatchSource:0}: Error finding container ce243599dd3f8161745c8962ec143bdccfae207b3988778c8a5a521cf1eca551: Status 404 returned error can't find the container with id ce243599dd3f8161745c8962ec143bdccfae207b3988778c8a5a521cf1eca551
Jul 26 18:39:41.120129 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:41.120094    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7795-140293206599552"
Jul 26 18:39:41.120321 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:41.120146    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0a1024f0e83b12df67ec8e88422cb99dc6cf7466b7074aa6ae9a24424ff3371f"
Jul 26 18:39:41.121325 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:41.121296    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7795-140293211309664" event={"ID":"37b26d07-144d-4d7e-ba8a-d4327b760ab2","Type":"ContainerStarted","Data":"ce243599dd3f8161745c8962ec143bdccfae207b3988778c8a5a521cf1eca551"}
Jul 26 18:39:41.130903 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:39:41.130879    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7795-140293206599552\" is forbidden: User \"system:node:ip-10-0-128-45.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-45.ec2.internal' and this object" podUID="b17b0d5c-1510-4c5b-8b1d-86e64b423ae8" pod="opendatahub/test-curl-7795-140293206599552"
Jul 26 18:39:41.839223 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:41.839193    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b17b0d5c-1510-4c5b-8b1d-86e64b423ae8" path="/var/lib/kubelet/pods/b17b0d5c-1510-4c5b-8b1d-86e64b423ae8/volumes"
Jul 26 18:39:42.004748 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:42.004700    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659" podUID="81d393f9-4c2f-46db-8456-02db554aab93" containerName="main" containerID="cri-o://fcb1589ac1d9091041187887ed0c9ab7b1edbce0585f75695f3f7d22a44d9852" gracePeriod=45
Jul 26 18:39:42.129516 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:42.129485    2570 generic.go:356] "Generic (PLEG): container finished" podID="81d393f9-4c2f-46db-8456-02db554aab93" containerID="fcb1589ac1d9091041187887ed0c9ab7b1edbce0585f75695f3f7d22a44d9852" exitCode=0
Jul 26 18:39:42.129666 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:42.129552    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659" event={"ID":"81d393f9-4c2f-46db-8456-02db554aab93","Type":"ContainerDied","Data":"fcb1589ac1d9091041187887ed0c9ab7b1edbce0585f75695f3f7d22a44d9852"}
Jul 26 18:39:42.131101 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:42.131080    2570 generic.go:356] "Generic (PLEG): container finished" podID="37b26d07-144d-4d7e-ba8a-d4327b760ab2" containerID="cdcf98c1a163f383971abce27540a0f3fe8de3349403a8441dabc70a9e19fef7" exitCode=0
Jul 26 18:39:42.131262 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:42.131142    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7795-140293211309664" event={"ID":"37b26d07-144d-4d7e-ba8a-d4327b760ab2","Type":"ContainerDied","Data":"cdcf98c1a163f383971abce27540a0f3fe8de3349403a8441dabc70a9e19fef7"}
Jul 26 18:39:42.257200 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:42.256965    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:39:42.427434 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:42.427337    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81d393f9-4c2f-46db-8456-02db554aab93-dshm\") pod \"81d393f9-4c2f-46db-8456-02db554aab93\" (UID: \"81d393f9-4c2f-46db-8456-02db554aab93\") "
Jul 26 18:39:42.427434 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:42.427376    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81d393f9-4c2f-46db-8456-02db554aab93-model-cache\") pod \"81d393f9-4c2f-46db-8456-02db554aab93\" (UID: \"81d393f9-4c2f-46db-8456-02db554aab93\") "
Jul 26 18:39:42.427434 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:42.427400    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81d393f9-4c2f-46db-8456-02db554aab93-home\") pod \"81d393f9-4c2f-46db-8456-02db554aab93\" (UID: \"81d393f9-4c2f-46db-8456-02db554aab93\") "
Jul 26 18:39:42.427434 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:42.427426    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhfsh\" (UniqueName: \"kubernetes.io/projected/81d393f9-4c2f-46db-8456-02db554aab93-kube-api-access-hhfsh\") pod \"81d393f9-4c2f-46db-8456-02db554aab93\" (UID: \"81d393f9-4c2f-46db-8456-02db554aab93\") "
Jul 26 18:39:42.427777 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:42.427460    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81d393f9-4c2f-46db-8456-02db554aab93-tls-certs\") pod \"81d393f9-4c2f-46db-8456-02db554aab93\" (UID: \"81d393f9-4c2f-46db-8456-02db554aab93\") "
Jul 26 18:39:42.427777 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:42.427491    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81d393f9-4c2f-46db-8456-02db554aab93-tmp-dir\") pod \"81d393f9-4c2f-46db-8456-02db554aab93\" (UID: \"81d393f9-4c2f-46db-8456-02db554aab93\") "
Jul 26 18:39:42.427777 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:42.427678    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81d393f9-4c2f-46db-8456-02db554aab93-model-cache" (OuterVolumeSpecName: "model-cache") pod "81d393f9-4c2f-46db-8456-02db554aab93" (UID: "81d393f9-4c2f-46db-8456-02db554aab93"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:39:42.427941 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:42.427798    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81d393f9-4c2f-46db-8456-02db554aab93-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "81d393f9-4c2f-46db-8456-02db554aab93" (UID: "81d393f9-4c2f-46db-8456-02db554aab93"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:39:42.427941 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:42.427827    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81d393f9-4c2f-46db-8456-02db554aab93-model-cache\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:39:42.428043 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:42.428004    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81d393f9-4c2f-46db-8456-02db554aab93-home" (OuterVolumeSpecName: "home") pod "81d393f9-4c2f-46db-8456-02db554aab93" (UID: "81d393f9-4c2f-46db-8456-02db554aab93"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:39:42.429708 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:42.429681    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81d393f9-4c2f-46db-8456-02db554aab93-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "81d393f9-4c2f-46db-8456-02db554aab93" (UID: "81d393f9-4c2f-46db-8456-02db554aab93"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:39:42.429708 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:42.429698    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81d393f9-4c2f-46db-8456-02db554aab93-dshm" (OuterVolumeSpecName: "dshm") pod "81d393f9-4c2f-46db-8456-02db554aab93" (UID: "81d393f9-4c2f-46db-8456-02db554aab93"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:39:42.429861 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:42.429689    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81d393f9-4c2f-46db-8456-02db554aab93-kube-api-access-hhfsh" (OuterVolumeSpecName: "kube-api-access-hhfsh") pod "81d393f9-4c2f-46db-8456-02db554aab93" (UID: "81d393f9-4c2f-46db-8456-02db554aab93"). InnerVolumeSpecName "kube-api-access-hhfsh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:39:42.528709 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:42.528679    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81d393f9-4c2f-46db-8456-02db554aab93-dshm\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:39:42.528709 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:42.528705    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81d393f9-4c2f-46db-8456-02db554aab93-home\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:39:42.528709 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:42.528715    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhfsh\" (UniqueName: \"kubernetes.io/projected/81d393f9-4c2f-46db-8456-02db554aab93-kube-api-access-hhfsh\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:39:42.528914 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:42.528724    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81d393f9-4c2f-46db-8456-02db554aab93-tls-certs\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:39:42.528914 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:42.528733    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81d393f9-4c2f-46db-8456-02db554aab93-tmp-dir\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:39:43.138061 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.138027    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"
Jul 26 18:39:43.138532 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.138061    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659" event={"ID":"81d393f9-4c2f-46db-8456-02db554aab93","Type":"ContainerDied","Data":"6e7805cde9eb5acde49fac8a3e5c4039196ec3108f8ace7db7177af9fed2c096"}
Jul 26 18:39:43.138532 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.138125    2570 scope.go:117] "RemoveContainer" containerID="fcb1589ac1d9091041187887ed0c9ab7b1edbce0585f75695f3f7d22a44d9852"
Jul 26 18:39:43.159863 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.159836    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"]
Jul 26 18:39:43.163886 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.163859    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-6aa894/rate-test-model-6aa894-kserve-848bbdcccf-7q659"]
Jul 26 18:39:43.282729 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.282703    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7795-140293211309664"
Jul 26 18:39:43.301897 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.301873    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7795-140293211309664_37b26d07-144d-4d7e-ba8a-d4327b760ab2/test-curl-7795-140293211309664/0.log"
Jul 26 18:39:43.328068 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.328033    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7795-140293211309664"]
Jul 26 18:39:43.331090 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.331065    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7795-140293211309664"]
Jul 26 18:39:43.436370 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.436278    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnfbd\" (UniqueName: \"kubernetes.io/projected/37b26d07-144d-4d7e-ba8a-d4327b760ab2-kube-api-access-cnfbd\") pod \"37b26d07-144d-4d7e-ba8a-d4327b760ab2\" (UID: \"37b26d07-144d-4d7e-ba8a-d4327b760ab2\") "
Jul 26 18:39:43.438370 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.438346    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37b26d07-144d-4d7e-ba8a-d4327b760ab2-kube-api-access-cnfbd" (OuterVolumeSpecName: "kube-api-access-cnfbd") pod "37b26d07-144d-4d7e-ba8a-d4327b760ab2" (UID: "37b26d07-144d-4d7e-ba8a-d4327b760ab2"). InnerVolumeSpecName "kube-api-access-cnfbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:39:43.452467 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.452443    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7795-140293211309664"]
Jul 26 18:39:43.452824 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.452812    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37b26d07-144d-4d7e-ba8a-d4327b760ab2" containerName="test-curl-7795-140293211309664"
Jul 26 18:39:43.452869 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.452826    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="37b26d07-144d-4d7e-ba8a-d4327b760ab2" containerName="test-curl-7795-140293211309664"
Jul 26 18:39:43.452869 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.452842    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81d393f9-4c2f-46db-8456-02db554aab93" containerName="main"
Jul 26 18:39:43.452869 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.452848    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="81d393f9-4c2f-46db-8456-02db554aab93" containerName="main"
Jul 26 18:39:43.452961 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.452903    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="37b26d07-144d-4d7e-ba8a-d4327b760ab2" containerName="test-curl-7795-140293211309664"
Jul 26 18:39:43.452961 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.452916    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="81d393f9-4c2f-46db-8456-02db554aab93" containerName="main"
Jul 26 18:39:43.454889 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.454873    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7795-140293211309664"
Jul 26 18:39:43.462846 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.462681    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7795-140293211309664"]
Jul 26 18:39:43.537802 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.537772    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m4pz\" (UniqueName: \"kubernetes.io/projected/401b6ae3-e012-4426-84b6-b4ca1646c5cd-kube-api-access-7m4pz\") pod \"test-curl-7795-140293211309664\" (UID: \"401b6ae3-e012-4426-84b6-b4ca1646c5cd\") " pod="opendatahub/test-curl-7795-140293211309664"
Jul 26 18:39:43.537993 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.537841    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnfbd\" (UniqueName: \"kubernetes.io/projected/37b26d07-144d-4d7e-ba8a-d4327b760ab2-kube-api-access-cnfbd\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:39:43.638933 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.638891    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m4pz\" (UniqueName: \"kubernetes.io/projected/401b6ae3-e012-4426-84b6-b4ca1646c5cd-kube-api-access-7m4pz\") pod \"test-curl-7795-140293211309664\" (UID: \"401b6ae3-e012-4426-84b6-b4ca1646c5cd\") " pod="opendatahub/test-curl-7795-140293211309664"
Jul 26 18:39:43.647017 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.646987    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m4pz\" (UniqueName: \"kubernetes.io/projected/401b6ae3-e012-4426-84b6-b4ca1646c5cd-kube-api-access-7m4pz\") pod \"test-curl-7795-140293211309664\" (UID: \"401b6ae3-e012-4426-84b6-b4ca1646c5cd\") " pod="opendatahub/test-curl-7795-140293211309664"
Jul 26 18:39:43.765679 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.765646    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7795-140293211309664"
Jul 26 18:39:43.840572 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.840538    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37b26d07-144d-4d7e-ba8a-d4327b760ab2" path="/var/lib/kubelet/pods/37b26d07-144d-4d7e-ba8a-d4327b760ab2/volumes"
Jul 26 18:39:43.841149 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.841098    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81d393f9-4c2f-46db-8456-02db554aab93" path="/var/lib/kubelet/pods/81d393f9-4c2f-46db-8456-02db554aab93/volumes"
Jul 26 18:39:43.901764 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:43.901737    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7795-140293211309664"]
Jul 26 18:39:43.903505 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:39:43.903478    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod401b6ae3_e012_4426_84b6_b4ca1646c5cd.slice/crio-934cf6a372dddeb63767e3ee368337864484b2c88ae75eb7bae5bc244dc66988 WatchSource:0}: Error finding container 934cf6a372dddeb63767e3ee368337864484b2c88ae75eb7bae5bc244dc66988: Status 404 returned error can't find the container with id 934cf6a372dddeb63767e3ee368337864484b2c88ae75eb7bae5bc244dc66988
Jul 26 18:39:44.144497 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:44.144403    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7795-140293211309664" event={"ID":"401b6ae3-e012-4426-84b6-b4ca1646c5cd","Type":"ContainerStarted","Data":"934cf6a372dddeb63767e3ee368337864484b2c88ae75eb7bae5bc244dc66988"}
Jul 26 18:39:44.145759 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:44.145738    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7795-140293211309664"
Jul 26 18:39:44.145885 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:44.145783    2570 scope.go:117] "RemoveContainer" containerID="cdcf98c1a163f383971abce27540a0f3fe8de3349403a8441dabc70a9e19fef7"
Jul 26 18:39:45.154797 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:45.154762    2570 generic.go:356] "Generic (PLEG): container finished" podID="401b6ae3-e012-4426-84b6-b4ca1646c5cd" containerID="e5b4ab04ba2348aad794d3a08b610967a7f99c707dc75ea901f23f22951e98f4" exitCode=0
Jul 26 18:39:45.155253 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:45.154854    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7795-140293211309664" event={"ID":"401b6ae3-e012-4426-84b6-b4ca1646c5cd","Type":"ContainerDied","Data":"e5b4ab04ba2348aad794d3a08b610967a7f99c707dc75ea901f23f22951e98f4"}
Jul 26 18:39:46.297445 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:46.297421    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7795-140293211309664"
Jul 26 18:39:46.315685 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:46.315658    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7795-140293211309664_401b6ae3-e012-4426-84b6-b4ca1646c5cd/test-curl-7795-140293211309664/0.log"
Jul 26 18:39:46.337584 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:46.337557    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7795-140293211309664"]
Jul 26 18:39:46.341518 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:46.341493    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7795-140293211309664"]
Jul 26 18:39:46.464487 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:46.464412    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7m4pz\" (UniqueName: \"kubernetes.io/projected/401b6ae3-e012-4426-84b6-b4ca1646c5cd-kube-api-access-7m4pz\") pod \"401b6ae3-e012-4426-84b6-b4ca1646c5cd\" (UID: \"401b6ae3-e012-4426-84b6-b4ca1646c5cd\") "
Jul 26 18:39:46.466438 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:46.466416    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/401b6ae3-e012-4426-84b6-b4ca1646c5cd-kube-api-access-7m4pz" (OuterVolumeSpecName: "kube-api-access-7m4pz") pod "401b6ae3-e012-4426-84b6-b4ca1646c5cd" (UID: "401b6ae3-e012-4426-84b6-b4ca1646c5cd"). InnerVolumeSpecName "kube-api-access-7m4pz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:39:46.565206 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:46.565157    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7m4pz\" (UniqueName: \"kubernetes.io/projected/401b6ae3-e012-4426-84b6-b4ca1646c5cd-kube-api-access-7m4pz\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:39:47.171385 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:47.171342    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="934cf6a372dddeb63767e3ee368337864484b2c88ae75eb7bae5bc244dc66988"
Jul 26 18:39:47.171543 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:47.171389    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7795-140293211309664"
Jul 26 18:39:47.182315 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:39:47.182288    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7795-140293211309664\" is forbidden: User \"system:node:ip-10-0-128-45.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-45.ec2.internal' and this object" podUID="401b6ae3-e012-4426-84b6-b4ca1646c5cd" pod="opendatahub/test-curl-7795-140293211309664"
Jul 26 18:39:47.838641 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:47.838607    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="401b6ae3-e012-4426-84b6-b4ca1646c5cd" path="/var/lib/kubelet/pods/401b6ae3-e012-4426-84b6-b4ca1646c5cd/volumes"
Jul 26 18:39:50.256645 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:50.256606    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk"]
Jul 26 18:39:50.257054 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:50.256971    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="401b6ae3-e012-4426-84b6-b4ca1646c5cd" containerName="test-curl-7795-140293211309664"
Jul 26 18:39:50.257054 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:50.256983    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="401b6ae3-e012-4426-84b6-b4ca1646c5cd" containerName="test-curl-7795-140293211309664"
Jul 26 18:39:50.257054 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:50.257054    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="401b6ae3-e012-4426-84b6-b4ca1646c5cd" containerName="test-curl-7795-140293211309664"
Jul 26 18:39:50.259090 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:50.259073    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk"
Jul 26 18:39:50.265283 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:50.265263    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-76b936-serving-cert\""
Jul 26 18:39:50.288163 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:50.288128    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk"]
Jul 26 18:39:50.397530 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:50.397494    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/806fa5fa-a85e-4d5c-9599-67e49291e2f3-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-76b936-768f675c59-77wgk\" (UID: \"806fa5fa-a85e-4d5c-9599-67e49291e2f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk"
Jul 26 18:39:50.397715 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:50.397542    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt84b\" (UniqueName: \"kubernetes.io/projected/806fa5fa-a85e-4d5c-9599-67e49291e2f3-kube-api-access-mt84b\") pod \"maas-api-e2e-ipp-a-76b936-768f675c59-77wgk\" (UID: \"806fa5fa-a85e-4d5c-9599-67e49291e2f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk"
Jul 26 18:39:50.397715 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:50.397633    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/806fa5fa-a85e-4d5c-9599-67e49291e2f3-maas-api-tls\") pod \"maas-api-e2e-ipp-a-76b936-768f675c59-77wgk\" (UID: \"806fa5fa-a85e-4d5c-9599-67e49291e2f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk"
Jul 26 18:39:50.498583 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:50.498541    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/806fa5fa-a85e-4d5c-9599-67e49291e2f3-maas-api-tls\") pod \"maas-api-e2e-ipp-a-76b936-768f675c59-77wgk\" (UID: \"806fa5fa-a85e-4d5c-9599-67e49291e2f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk"
Jul 26 18:39:50.498775 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:50.498620    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/806fa5fa-a85e-4d5c-9599-67e49291e2f3-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-76b936-768f675c59-77wgk\" (UID: \"806fa5fa-a85e-4d5c-9599-67e49291e2f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk"
Jul 26 18:39:50.498775 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:50.498658    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt84b\" (UniqueName: \"kubernetes.io/projected/806fa5fa-a85e-4d5c-9599-67e49291e2f3-kube-api-access-mt84b\") pod \"maas-api-e2e-ipp-a-76b936-768f675c59-77wgk\" (UID: \"806fa5fa-a85e-4d5c-9599-67e49291e2f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk"
Jul 26 18:39:50.499350 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:50.499324    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/806fa5fa-a85e-4d5c-9599-67e49291e2f3-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-76b936-768f675c59-77wgk\" (UID: \"806fa5fa-a85e-4d5c-9599-67e49291e2f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk"
Jul 26 18:39:50.501064 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:50.501042    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/806fa5fa-a85e-4d5c-9599-67e49291e2f3-maas-api-tls\") pod \"maas-api-e2e-ipp-a-76b936-768f675c59-77wgk\" (UID: \"806fa5fa-a85e-4d5c-9599-67e49291e2f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk"
Jul 26 18:39:50.513840 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:50.513806    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt84b\" (UniqueName: \"kubernetes.io/projected/806fa5fa-a85e-4d5c-9599-67e49291e2f3-kube-api-access-mt84b\") pod \"maas-api-e2e-ipp-a-76b936-768f675c59-77wgk\" (UID: \"806fa5fa-a85e-4d5c-9599-67e49291e2f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk"
Jul 26 18:39:50.569566 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:50.569533    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk"
Jul 26 18:39:50.754054 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:50.754025    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk"]
Jul 26 18:39:50.755947 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:39:50.755924    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod806fa5fa_a85e_4d5c_9599_67e49291e2f3.slice/crio-a2bc0ae8bf58200cbda349d061e12f594f453f8d9c966e00cad55236edeae25f WatchSource:0}: Error finding container a2bc0ae8bf58200cbda349d061e12f594f453f8d9c966e00cad55236edeae25f: Status 404 returned error can't find the container with id a2bc0ae8bf58200cbda349d061e12f594f453f8d9c966e00cad55236edeae25f
Jul 26 18:39:51.194712 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:51.194677    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk" event={"ID":"806fa5fa-a85e-4d5c-9599-67e49291e2f3","Type":"ContainerStarted","Data":"a2bc0ae8bf58200cbda349d061e12f594f453f8d9c966e00cad55236edeae25f"}
Jul 26 18:39:52.201737 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:52.201699    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk" event={"ID":"806fa5fa-a85e-4d5c-9599-67e49291e2f3","Type":"ContainerStarted","Data":"e22c66fd6a7108006687dfce20b1b027cf746cb7e3ff9274f1900414f30ce9f4"}
Jul 26 18:39:52.202151 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:52.201876    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk"
Jul 26 18:39:52.238727 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:52.238657    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk" podStartSLOduration=1.682774679 podStartE2EDuration="2.238636182s" totalImagesPullingTime="555.861503ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:39:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:39:52.238380357 +0000 UTC m=+3048.890571214" watchObservedRunningTime="2026-07-26 18:39:52.238636182 +0000 UTC m=+3048.890827041"
Jul 26 18:39:58.212996 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:39:58.212920    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk"
Jul 26 18:40:09.004582 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:09.004544    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82"]
Jul 26 18:40:09.008760 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:09.008738    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82"
Jul 26 18:40:09.011261 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:09.011235    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-3e8329-serving-cert\""
Jul 26 18:40:09.018193 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:09.018165    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82"]
Jul 26 18:40:09.051065 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:09.051034    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/005b8b98-b1f3-494a-887a-40ec533c1a46-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82\" (UID: \"005b8b98-b1f3-494a-887a-40ec533c1a46\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82"
Jul 26 18:40:09.051275 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:09.051100    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/005b8b98-b1f3-494a-887a-40ec533c1a46-maas-api-tls\") pod \"maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82\" (UID: \"005b8b98-b1f3-494a-887a-40ec533c1a46\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82"
Jul 26 18:40:09.051275 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:09.051214    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn7jl\" (UniqueName: \"kubernetes.io/projected/005b8b98-b1f3-494a-887a-40ec533c1a46-kube-api-access-fn7jl\") pod \"maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82\" (UID: \"005b8b98-b1f3-494a-887a-40ec533c1a46\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82"
Jul 26 18:40:09.152034 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:09.151991    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/005b8b98-b1f3-494a-887a-40ec533c1a46-maas-api-tls\") pod \"maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82\" (UID: \"005b8b98-b1f3-494a-887a-40ec533c1a46\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82"
Jul 26 18:40:09.152250 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:09.152066    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn7jl\" (UniqueName: \"kubernetes.io/projected/005b8b98-b1f3-494a-887a-40ec533c1a46-kube-api-access-fn7jl\") pod \"maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82\" (UID: \"005b8b98-b1f3-494a-887a-40ec533c1a46\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82"
Jul 26 18:40:09.152250 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:09.152140    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/005b8b98-b1f3-494a-887a-40ec533c1a46-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82\" (UID: \"005b8b98-b1f3-494a-887a-40ec533c1a46\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82"
Jul 26 18:40:09.152250 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:40:09.152192    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-serving-cert: secret "maas-api-e2e-ipp-b-3e8329-serving-cert" not found
Jul 26 18:40:09.152373 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:40:09.152282    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/005b8b98-b1f3-494a-887a-40ec533c1a46-maas-api-tls podName:005b8b98-b1f3-494a-887a-40ec533c1a46 nodeName:}" failed. No retries permitted until 2026-07-26 18:40:09.652259847 +0000 UTC m=+3066.304450687 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/005b8b98-b1f3-494a-887a-40ec533c1a46-maas-api-tls") pod "maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82" (UID: "005b8b98-b1f3-494a-887a-40ec533c1a46") : secret "maas-api-e2e-ipp-b-3e8329-serving-cert" not found
Jul 26 18:40:09.152842 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:09.152822    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/005b8b98-b1f3-494a-887a-40ec533c1a46-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82\" (UID: \"005b8b98-b1f3-494a-887a-40ec533c1a46\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82"
Jul 26 18:40:09.160504 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:09.160483    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn7jl\" (UniqueName: \"kubernetes.io/projected/005b8b98-b1f3-494a-887a-40ec533c1a46-kube-api-access-fn7jl\") pod \"maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82\" (UID: \"005b8b98-b1f3-494a-887a-40ec533c1a46\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82"
Jul 26 18:40:09.656653 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:09.656616    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/005b8b98-b1f3-494a-887a-40ec533c1a46-maas-api-tls\") pod \"maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82\" (UID: \"005b8b98-b1f3-494a-887a-40ec533c1a46\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82"
Jul 26 18:40:09.659048 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:09.659020    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/005b8b98-b1f3-494a-887a-40ec533c1a46-maas-api-tls\") pod \"maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82\" (UID: \"005b8b98-b1f3-494a-887a-40ec533c1a46\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82"
Jul 26 18:40:09.922616 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:09.922528    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82"
Jul 26 18:40:10.060234 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:10.060200    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82"]
Jul 26 18:40:10.061571 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:40:10.061542    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod005b8b98_b1f3_494a_887a_40ec533c1a46.slice/crio-2a78333849805c4bee4a907912bdea26cdaf63697a2e7b997422d269404a5900 WatchSource:0}: Error finding container 2a78333849805c4bee4a907912bdea26cdaf63697a2e7b997422d269404a5900: Status 404 returned error can't find the container with id 2a78333849805c4bee4a907912bdea26cdaf63697a2e7b997422d269404a5900
Jul 26 18:40:10.302455 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:10.302422    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82" event={"ID":"005b8b98-b1f3-494a-887a-40ec533c1a46","Type":"ContainerStarted","Data":"2a78333849805c4bee4a907912bdea26cdaf63697a2e7b997422d269404a5900"}
Jul 26 18:40:11.309447 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:11.309414    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82" event={"ID":"005b8b98-b1f3-494a-887a-40ec533c1a46","Type":"ContainerStarted","Data":"42da71bd7f4e73d1d1b73810b57cbf27ff1971b259e9a2eda3b9c18e6e1f8f09"}
Jul 26 18:40:11.309838 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:11.309485    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82"
Jul 26 18:40:11.331659 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:11.331608    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82" podStartSLOduration=2.952932993 podStartE2EDuration="3.331590357s" totalImagesPullingTime="378.657364ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:40:11.32971687 +0000 UTC m=+3067.981907728" watchObservedRunningTime="2026-07-26 18:40:11.331590357 +0000 UTC m=+3067.983781215"
Jul 26 18:40:17.320776 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:17.320743    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82"
Jul 26 18:40:30.501508 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:30.501477    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-fw8n6_22c0748d-5cd0-4d65-addb-2e9f3837a71d/payload-processing/0.log"
Jul 26 18:40:45.212847 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.212809    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"]
Jul 26 18:40:45.216173 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.216154    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:40:45.218453 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.218425    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-76b936\"/\"openshift-service-ca.crt\""
Jul 26 18:40:45.218600 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.218508    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-76b936\"/\"ipp-route-76b936-kserve-self-signed-certs\""
Jul 26 18:40:45.219126 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.219100    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-76b936\"/\"kube-root-ca.crt\""
Jul 26 18:40:45.219360 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.219347    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-76b936\"/\"default-dockercfg-md4nj\""
Jul 26 18:40:45.227490 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.227465    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"]
Jul 26 18:40:45.285355 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.285322    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-dshm\") pod \"ipp-route-76b936-kserve-787664fcbc-tjvb7\" (UID: \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\") " pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:40:45.285514 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.285363    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-tmp-dir\") pod \"ipp-route-76b936-kserve-787664fcbc-tjvb7\" (UID: \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\") " pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:40:45.285514 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.285452    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-home\") pod \"ipp-route-76b936-kserve-787664fcbc-tjvb7\" (UID: \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\") " pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:40:45.285603 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.285511    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-model-cache\") pod \"ipp-route-76b936-kserve-787664fcbc-tjvb7\" (UID: \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\") " pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:40:45.285603 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.285539    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft567\" (UniqueName: \"kubernetes.io/projected/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-kube-api-access-ft567\") pod \"ipp-route-76b936-kserve-787664fcbc-tjvb7\" (UID: \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\") " pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:40:45.285603 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.285570    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-tls-certs\") pod \"ipp-route-76b936-kserve-787664fcbc-tjvb7\" (UID: \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\") " pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:40:45.386703 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.386662    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-model-cache\") pod \"ipp-route-76b936-kserve-787664fcbc-tjvb7\" (UID: \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\") " pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:40:45.386703 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.386702    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft567\" (UniqueName: \"kubernetes.io/projected/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-kube-api-access-ft567\") pod \"ipp-route-76b936-kserve-787664fcbc-tjvb7\" (UID: \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\") " pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:40:45.386967 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.386725    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-tls-certs\") pod \"ipp-route-76b936-kserve-787664fcbc-tjvb7\" (UID: \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\") " pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:40:45.386967 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.386763    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-dshm\") pod \"ipp-route-76b936-kserve-787664fcbc-tjvb7\" (UID: \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\") " pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:40:45.386967 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.386779    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-tmp-dir\") pod \"ipp-route-76b936-kserve-787664fcbc-tjvb7\" (UID: \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\") " pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:40:45.386967 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.386933    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-home\") pod \"ipp-route-76b936-kserve-787664fcbc-tjvb7\" (UID: \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\") " pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:40:45.387232 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.387187    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-tmp-dir\") pod \"ipp-route-76b936-kserve-787664fcbc-tjvb7\" (UID: \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\") " pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:40:45.387327 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.387288    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-model-cache\") pod \"ipp-route-76b936-kserve-787664fcbc-tjvb7\" (UID: \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\") " pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:40:45.387327 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.387306    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-home\") pod \"ipp-route-76b936-kserve-787664fcbc-tjvb7\" (UID: \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\") " pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:40:45.389191 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.389164    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-dshm\") pod \"ipp-route-76b936-kserve-787664fcbc-tjvb7\" (UID: \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\") " pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:40:45.389397 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.389380    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-tls-certs\") pod \"ipp-route-76b936-kserve-787664fcbc-tjvb7\" (UID: \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\") " pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:40:45.395049 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.395027    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft567\" (UniqueName: \"kubernetes.io/projected/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-kube-api-access-ft567\") pod \"ipp-route-76b936-kserve-787664fcbc-tjvb7\" (UID: \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\") " pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:40:45.526760 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.526727    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:40:45.665954 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:45.665921    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"]
Jul 26 18:40:45.667083 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:40:45.667055    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07bb2b32_5d3c_47b7_9998_f71e03c5a17e.slice/crio-961382d6b6634b879a840ddf7ca0a002c3b49e538aaf8aa005bc8a14d0c95246 WatchSource:0}: Error finding container 961382d6b6634b879a840ddf7ca0a002c3b49e538aaf8aa005bc8a14d0c95246: Status 404 returned error can't find the container with id 961382d6b6634b879a840ddf7ca0a002c3b49e538aaf8aa005bc8a14d0c95246
Jul 26 18:40:46.501170 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:46.501128    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7" event={"ID":"07bb2b32-5d3c-47b7-9998-f71e03c5a17e","Type":"ContainerStarted","Data":"444b8f3c73a1403fe7e489b25a3cbf3b12494ade70c88bae4d6dc33d465e5946"}
Jul 26 18:40:46.501540 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:46.501178    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7" event={"ID":"07bb2b32-5d3c-47b7-9998-f71e03c5a17e","Type":"ContainerStarted","Data":"961382d6b6634b879a840ddf7ca0a002c3b49e538aaf8aa005bc8a14d0c95246"}
Jul 26 18:40:46.520817 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:46.520774    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7" podStartSLOduration=1.5207610790000001 podStartE2EDuration="1.520761079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:40:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:40:46.518538463 +0000 UTC m=+3103.170729321" watchObservedRunningTime="2026-07-26 18:40:46.520761079 +0000 UTC m=+3103.172951936"
Jul 26 18:40:55.526975 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:55.526937    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:40:55.527412 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:55.526996    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:40:55.539537 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:55.539510    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:40:55.561849 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:40:55.561821    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:41:27.819101 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:27.818989    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-fw8n6_22c0748d-5cd0-4d65-addb-2e9f3837a71d/payload-processing/0.log"
Jul 26 18:41:28.498953 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.498917    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"]
Jul 26 18:41:28.502621 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.502599    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.505801 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.505775    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 26 18:41:28.506030 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.506014    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-4147f2-openshift-default-dockercfg-pwcb7\""
Jul 26 18:41:28.517225 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.517202    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"]
Jul 26 18:41:28.666134 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.666078    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istio-podinfo\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.666317 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.666147    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istio-token\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.666317 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.666249    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istiod-ca-cert\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.666317 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.666298    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istio-envoy\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.666437 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.666318    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istio-data\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.666437 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.666369    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-workload-socket\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.666437 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.666392    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-workload-certs\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.666437 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.666421    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rb2x\" (UniqueName: \"kubernetes.io/projected/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-kube-api-access-5rb2x\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.666566 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.666492    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-credential-socket\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.767504 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.767471    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rb2x\" (UniqueName: \"kubernetes.io/projected/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-kube-api-access-5rb2x\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.767663 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.767518    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-credential-socket\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.767663 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.767578    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istio-podinfo\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.767663 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.767607    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istio-token\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.767663 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.767656    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istiod-ca-cert\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.767877 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.767702    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istio-envoy\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.767877 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.767727    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istio-data\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.767877 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.767769    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-workload-socket\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.767877 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.767791    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-workload-certs\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.768087 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.768007    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-credential-socket\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.768198 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.768175    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-workload-certs\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.768289 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.768266    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istio-data\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.768341 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.768297    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-workload-socket\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.768519 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.768488    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istiod-ca-cert\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.770067 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.770042    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istio-envoy\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.770315 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.770298    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istio-podinfo\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.776500 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.776476    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istio-token\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.776638 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.776618    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rb2x\" (UniqueName: \"kubernetes.io/projected/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-kube-api-access-5rb2x\") pod \"e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") " pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.812579 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.812549    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:28.972174 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:28.972147    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"]
Jul 26 18:41:28.975189 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:41:28.975148    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33dfe2f4_147e_44a9_9d50_e7ac38670fd8.slice/crio-7a4e31ea3537e43af2b5139c75ef8307538dfbd358f18d4f642bfc09a59892f0 WatchSource:0}: Error finding container 7a4e31ea3537e43af2b5139c75ef8307538dfbd358f18d4f642bfc09a59892f0: Status 404 returned error can't find the container with id 7a4e31ea3537e43af2b5139c75ef8307538dfbd358f18d4f642bfc09a59892f0
Jul 26 18:41:29.747757 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:29.747718    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m" event={"ID":"33dfe2f4-147e-44a9-9d50-e7ac38670fd8","Type":"ContainerStarted","Data":"7a4e31ea3537e43af2b5139c75ef8307538dfbd358f18d4f642bfc09a59892f0"}
Jul 26 18:41:31.601214 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:31.601179    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:41:31.601501 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:31.601248    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:41:31.601501 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:31.601279    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:41:31.763672 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:31.763624    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m" event={"ID":"33dfe2f4-147e-44a9-9d50-e7ac38670fd8","Type":"ContainerStarted","Data":"ccbc685793ffc30de7e4ae5c199d7d0458caca4d9bcce7c96e33e3f2c7932219"}
Jul 26 18:41:31.786709 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:31.786653    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m" podStartSLOduration=1.16393069 podStartE2EDuration="3.786637334s" totalImagesPullingTime="2.622706644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:41:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:41:31.785186985 +0000 UTC m=+3148.437377842" watchObservedRunningTime="2026-07-26 18:41:31.786637334 +0000 UTC m=+3148.438828194"
Jul 26 18:41:31.812985 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:31.812950    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:32.818176 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:32.818146    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:33.775747 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:33.775702    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:33.776918 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:33.776896    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:33.782657 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:33.782631    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj"]
Jul 26 18:41:33.786775 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:33.786756    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj"
Jul 26 18:41:33.788953 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:33.788934    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-4147f2-serving-cert\""
Jul 26 18:41:33.796931 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:33.796908    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj"]
Jul 26 18:41:33.919473 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:33.919437    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c42934d9-a5b1-4aff-9672-3d935f64f31f-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj\" (UID: \"c42934d9-a5b1-4aff-9672-3d935f64f31f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj"
Jul 26 18:41:33.919948 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:33.919496    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzjhd\" (UniqueName: \"kubernetes.io/projected/c42934d9-a5b1-4aff-9672-3d935f64f31f-kube-api-access-kzjhd\") pod \"maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj\" (UID: \"c42934d9-a5b1-4aff-9672-3d935f64f31f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj"
Jul 26 18:41:33.919948 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:33.919525    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c42934d9-a5b1-4aff-9672-3d935f64f31f-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj\" (UID: \"c42934d9-a5b1-4aff-9672-3d935f64f31f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj"
Jul 26 18:41:34.021011 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:34.020965    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c42934d9-a5b1-4aff-9672-3d935f64f31f-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj\" (UID: \"c42934d9-a5b1-4aff-9672-3d935f64f31f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj"
Jul 26 18:41:34.021247 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:34.021047    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzjhd\" (UniqueName: \"kubernetes.io/projected/c42934d9-a5b1-4aff-9672-3d935f64f31f-kube-api-access-kzjhd\") pod \"maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj\" (UID: \"c42934d9-a5b1-4aff-9672-3d935f64f31f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj"
Jul 26 18:41:34.021247 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:34.021080    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c42934d9-a5b1-4aff-9672-3d935f64f31f-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj\" (UID: \"c42934d9-a5b1-4aff-9672-3d935f64f31f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj"
Jul 26 18:41:34.021247 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:41:34.021232    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-serving-cert: secret "maas-api-e2e-ipp-cleanup-4147f2-serving-cert" not found
Jul 26 18:41:34.021442 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:41:34.021302    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c42934d9-a5b1-4aff-9672-3d935f64f31f-maas-api-tls podName:c42934d9-a5b1-4aff-9672-3d935f64f31f nodeName:}" failed. No retries permitted until 2026-07-26 18:41:34.521280586 +0000 UTC m=+3151.173471430 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c42934d9-a5b1-4aff-9672-3d935f64f31f-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj" (UID: "c42934d9-a5b1-4aff-9672-3d935f64f31f") : secret "maas-api-e2e-ipp-cleanup-4147f2-serving-cert" not found
Jul 26 18:41:34.021750 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:34.021724    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c42934d9-a5b1-4aff-9672-3d935f64f31f-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj\" (UID: \"c42934d9-a5b1-4aff-9672-3d935f64f31f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj"
Jul 26 18:41:34.029843 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:34.029790    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzjhd\" (UniqueName: \"kubernetes.io/projected/c42934d9-a5b1-4aff-9672-3d935f64f31f-kube-api-access-kzjhd\") pod \"maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj\" (UID: \"c42934d9-a5b1-4aff-9672-3d935f64f31f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj"
Jul 26 18:41:34.525651 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:34.525619    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c42934d9-a5b1-4aff-9672-3d935f64f31f-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj\" (UID: \"c42934d9-a5b1-4aff-9672-3d935f64f31f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj"
Jul 26 18:41:34.528365 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:34.528336    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c42934d9-a5b1-4aff-9672-3d935f64f31f-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj\" (UID: \"c42934d9-a5b1-4aff-9672-3d935f64f31f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj"
Jul 26 18:41:34.698765 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:34.698731    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj"
Jul 26 18:41:35.068049 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:35.068024    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj"]
Jul 26 18:41:35.070204 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:41:35.070168    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc42934d9_a5b1_4aff_9672_3d935f64f31f.slice/crio-eaa97ce633130a12c96039719d13adab4ecb57a3f4421cf8d41427aa1064eb85 WatchSource:0}: Error finding container eaa97ce633130a12c96039719d13adab4ecb57a3f4421cf8d41427aa1064eb85: Status 404 returned error can't find the container with id eaa97ce633130a12c96039719d13adab4ecb57a3f4421cf8d41427aa1064eb85
Jul 26 18:41:35.787564 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:35.787517    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj" event={"ID":"c42934d9-a5b1-4aff-9672-3d935f64f31f","Type":"ContainerStarted","Data":"606cdfebef1cc21a40740818cb7921f7e650c55799b9c6ec1591aac2dbd7150a"}
Jul 26 18:41:35.787742 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:35.787583    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj" event={"ID":"c42934d9-a5b1-4aff-9672-3d935f64f31f","Type":"ContainerStarted","Data":"eaa97ce633130a12c96039719d13adab4ecb57a3f4421cf8d41427aa1064eb85"}
Jul 26 18:41:35.787742 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:35.787611    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj"
Jul 26 18:41:35.807244 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:35.807188    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj" podStartSLOduration=2.434195479 podStartE2EDuration="2.807172221s" totalImagesPullingTime="372.976742ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:41:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:41:35.805405238 +0000 UTC m=+3152.457596097" watchObservedRunningTime="2026-07-26 18:41:35.807172221 +0000 UTC m=+3152.459363078"
Jul 26 18:41:41.798380 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:41.798351    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj"
Jul 26 18:41:49.106061 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:49.106015    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4147f2-drtjk"]
Jul 26 18:41:49.109779 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:49.109756    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4147f2-drtjk"
Jul 26 18:41:49.112146 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:49.112102    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-swzk5\""
Jul 26 18:41:49.120433 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:49.120404    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4147f2-drtjk"]
Jul 26 18:41:49.157966 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:49.157921    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c8ad964-8934-426f-b9b0-20b7024ea6b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-4147f2-drtjk\" (UID: \"9c8ad964-8934-426f-b9b0-20b7024ea6b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4147f2-drtjk"
Jul 26 18:41:49.259084 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:49.259044    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c8ad964-8934-426f-b9b0-20b7024ea6b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-4147f2-drtjk\" (UID: \"9c8ad964-8934-426f-b9b0-20b7024ea6b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4147f2-drtjk"
Jul 26 18:41:49.259744 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:49.259719    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c8ad964-8934-426f-b9b0-20b7024ea6b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-4147f2-drtjk\" (UID: \"9c8ad964-8934-426f-b9b0-20b7024ea6b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4147f2-drtjk"
Jul 26 18:41:49.422177 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:49.422085    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4147f2-drtjk"
Jul 26 18:41:49.532531 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:49.532333    2570 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": read tcp 10.132.0.2:39520->10.132.0.103:15021: read: connection reset by peer" start-of-body=
Jul 26 18:41:49.532531 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:49.532434    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m" podUID="33dfe2f4-147e-44a9-9d50-e7ac38670fd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": read tcp 10.132.0.2:39520->10.132.0.103:15021: read: connection reset by peer"
Jul 26 18:41:49.565219 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:49.565189    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4147f2-drtjk"]
Jul 26 18:41:49.613186 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:41:49.613144    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c8ad964_8934_426f_b9b0_20b7024ea6b2.slice/crio-efd12cc10ee9c7ca72288e1ddc2179f2c003cd8349fecdee1484eeaa01d26d69 WatchSource:0}: Error finding container efd12cc10ee9c7ca72288e1ddc2179f2c003cd8349fecdee1484eeaa01d26d69: Status 404 returned error can't find the container with id efd12cc10ee9c7ca72288e1ddc2179f2c003cd8349fecdee1484eeaa01d26d69
Jul 26 18:41:49.883627 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:49.883591    2570 generic.go:356] "Generic (PLEG): container finished" podID="33dfe2f4-147e-44a9-9d50-e7ac38670fd8" containerID="ccbc685793ffc30de7e4ae5c199d7d0458caca4d9bcce7c96e33e3f2c7932219" exitCode=0
Jul 26 18:41:49.883811 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:49.883664    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m" event={"ID":"33dfe2f4-147e-44a9-9d50-e7ac38670fd8","Type":"ContainerDied","Data":"ccbc685793ffc30de7e4ae5c199d7d0458caca4d9bcce7c96e33e3f2c7932219"}
Jul 26 18:41:49.884127 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:49.884094    2570 scope.go:117] "RemoveContainer" containerID="ccbc685793ffc30de7e4ae5c199d7d0458caca4d9bcce7c96e33e3f2c7932219"
Jul 26 18:41:49.885007 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:49.884984    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4147f2-drtjk" event={"ID":"9c8ad964-8934-426f-b9b0-20b7024ea6b2","Type":"ContainerStarted","Data":"efd12cc10ee9c7ca72288e1ddc2179f2c003cd8349fecdee1484eeaa01d26d69"}
Jul 26 18:41:49.885339 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:49.885315    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:41:49.885396 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:49.885382    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:41:49.885432 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:49.885411    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 26 18:41:50.891956 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:50.891922    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m" event={"ID":"33dfe2f4-147e-44a9-9d50-e7ac38670fd8","Type":"ContainerStarted","Data":"f5aa0de90dccf255abe3c92bb1ad1b412474a398afccd3ec148978b4080fd698"}
Jul 26 18:41:50.893521 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:50.893494    2570 generic.go:356] "Generic (PLEG): container finished" podID="9c8ad964-8934-426f-b9b0-20b7024ea6b2" containerID="e5e6a4a3bb7d10a46f3f950de1fc6fe0f8d618adc1765f453013425a71ff0e75" exitCode=0
Jul 26 18:41:50.893658 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:50.893577    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4147f2-drtjk" event={"ID":"9c8ad964-8934-426f-b9b0-20b7024ea6b2","Type":"ContainerDied","Data":"e5e6a4a3bb7d10a46f3f950de1fc6fe0f8d618adc1765f453013425a71ff0e75"}
Jul 26 18:41:51.812755 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:51.812722    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:51.812959 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:51.812877    2570 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Jul 26 18:41:51.812959 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:51.812932    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m" podUID="33dfe2f4-147e-44a9-9d50-e7ac38670fd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Jul 26 18:41:52.075972 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:52.075904    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4147f2-drtjk"
Jul 26 18:41:52.184856 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:52.184820    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c8ad964-8934-426f-b9b0-20b7024ea6b2-maas-api-service-ca\") pod \"9c8ad964-8934-426f-b9b0-20b7024ea6b2\" (UID: \"9c8ad964-8934-426f-b9b0-20b7024ea6b2\") "
Jul 26 18:41:52.185217 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:52.185194    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c8ad964-8934-426f-b9b0-20b7024ea6b2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9c8ad964-8934-426f-b9b0-20b7024ea6b2" (UID: "9c8ad964-8934-426f-b9b0-20b7024ea6b2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:41:52.286207 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:52.286172    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c8ad964-8934-426f-b9b0-20b7024ea6b2-maas-api-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:41:52.813739 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:52.813706    2570 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Jul 26 18:41:52.813912 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:52.813759    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m" podUID="33dfe2f4-147e-44a9-9d50-e7ac38670fd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Jul 26 18:41:52.906834 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:52.906800    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4147f2-drtjk"
Jul 26 18:41:52.906834 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:52.906826    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4147f2-drtjk" event={"ID":"9c8ad964-8934-426f-b9b0-20b7024ea6b2","Type":"ContainerDied","Data":"efd12cc10ee9c7ca72288e1ddc2179f2c003cd8349fecdee1484eeaa01d26d69"}
Jul 26 18:41:52.907051 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:52.906876    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="efd12cc10ee9c7ca72288e1ddc2179f2c003cd8349fecdee1484eeaa01d26d69"
Jul 26 18:41:53.812809 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:53.812777    2570 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Jul 26 18:41:53.813235 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:53.812830    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m" podUID="33dfe2f4-147e-44a9-9d50-e7ac38670fd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Jul 26 18:41:54.817287 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:54.817250    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:54.817761 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:54.817739    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:54.818544 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:54.818525    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:41:59.583209 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:59.583176    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj"]
Jul 26 18:41:59.583740 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:59.583395    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj" podUID="c42934d9-a5b1-4aff-9672-3d935f64f31f" containerName="maas-api" containerID="cri-o://606cdfebef1cc21a40740818cb7921f7e650c55799b9c6ec1591aac2dbd7150a" gracePeriod=30
Jul 26 18:41:59.840996 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:59.840926    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj"
Jul 26 18:41:59.950140 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:59.950079    2570 generic.go:356] "Generic (PLEG): container finished" podID="c42934d9-a5b1-4aff-9672-3d935f64f31f" containerID="606cdfebef1cc21a40740818cb7921f7e650c55799b9c6ec1591aac2dbd7150a" exitCode=0
Jul 26 18:41:59.950313 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:59.950147    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj" event={"ID":"c42934d9-a5b1-4aff-9672-3d935f64f31f","Type":"ContainerDied","Data":"606cdfebef1cc21a40740818cb7921f7e650c55799b9c6ec1591aac2dbd7150a"}
Jul 26 18:41:59.950313 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:59.950194    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj" event={"ID":"c42934d9-a5b1-4aff-9672-3d935f64f31f","Type":"ContainerDied","Data":"eaa97ce633130a12c96039719d13adab4ecb57a3f4421cf8d41427aa1064eb85"}
Jul 26 18:41:59.950313 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:59.950218    2570 scope.go:117] "RemoveContainer" containerID="606cdfebef1cc21a40740818cb7921f7e650c55799b9c6ec1591aac2dbd7150a"
Jul 26 18:41:59.950313 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:59.950151    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj"
Jul 26 18:41:59.957432 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:59.957342    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c42934d9-a5b1-4aff-9672-3d935f64f31f-openshift-service-ca\") pod \"c42934d9-a5b1-4aff-9672-3d935f64f31f\" (UID: \"c42934d9-a5b1-4aff-9672-3d935f64f31f\") "
Jul 26 18:41:59.957432 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:59.957371    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c42934d9-a5b1-4aff-9672-3d935f64f31f-maas-api-tls\") pod \"c42934d9-a5b1-4aff-9672-3d935f64f31f\" (UID: \"c42934d9-a5b1-4aff-9672-3d935f64f31f\") "
Jul 26 18:41:59.957793 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:59.957454    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzjhd\" (UniqueName: \"kubernetes.io/projected/c42934d9-a5b1-4aff-9672-3d935f64f31f-kube-api-access-kzjhd\") pod \"c42934d9-a5b1-4aff-9672-3d935f64f31f\" (UID: \"c42934d9-a5b1-4aff-9672-3d935f64f31f\") "
Jul 26 18:41:59.957793 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:59.957730    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c42934d9-a5b1-4aff-9672-3d935f64f31f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c42934d9-a5b1-4aff-9672-3d935f64f31f" (UID: "c42934d9-a5b1-4aff-9672-3d935f64f31f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:41:59.959864 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:59.959834    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c42934d9-a5b1-4aff-9672-3d935f64f31f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c42934d9-a5b1-4aff-9672-3d935f64f31f" (UID: "c42934d9-a5b1-4aff-9672-3d935f64f31f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:41:59.960030 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:59.960003    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c42934d9-a5b1-4aff-9672-3d935f64f31f-kube-api-access-kzjhd" (OuterVolumeSpecName: "kube-api-access-kzjhd") pod "c42934d9-a5b1-4aff-9672-3d935f64f31f" (UID: "c42934d9-a5b1-4aff-9672-3d935f64f31f"). InnerVolumeSpecName "kube-api-access-kzjhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:41:59.960425 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:59.960386    2570 scope.go:117] "RemoveContainer" containerID="606cdfebef1cc21a40740818cb7921f7e650c55799b9c6ec1591aac2dbd7150a"
Jul 26 18:41:59.960704 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:41:59.960686    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"606cdfebef1cc21a40740818cb7921f7e650c55799b9c6ec1591aac2dbd7150a\": container with ID starting with 606cdfebef1cc21a40740818cb7921f7e650c55799b9c6ec1591aac2dbd7150a not found: ID does not exist" containerID="606cdfebef1cc21a40740818cb7921f7e650c55799b9c6ec1591aac2dbd7150a"
Jul 26 18:41:59.960793 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:41:59.960717    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"606cdfebef1cc21a40740818cb7921f7e650c55799b9c6ec1591aac2dbd7150a"} err="failed to get container status \"606cdfebef1cc21a40740818cb7921f7e650c55799b9c6ec1591aac2dbd7150a\": rpc error: code = NotFound desc = could not find container \"606cdfebef1cc21a40740818cb7921f7e650c55799b9c6ec1591aac2dbd7150a\": container with ID starting with 606cdfebef1cc21a40740818cb7921f7e650c55799b9c6ec1591aac2dbd7150a not found: ID does not exist"
Jul 26 18:42:00.059179 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:00.059141    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzjhd\" (UniqueName: \"kubernetes.io/projected/c42934d9-a5b1-4aff-9672-3d935f64f31f-kube-api-access-kzjhd\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:42:00.059179 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:00.059174    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c42934d9-a5b1-4aff-9672-3d935f64f31f-openshift-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:42:00.059179 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:00.059185    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c42934d9-a5b1-4aff-9672-3d935f64f31f-maas-api-tls\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:42:00.274456 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:00.274413    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj"]
Jul 26 18:42:00.277524 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:00.277497    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4147f2-5d65db9d9c-m9xzj"]
Jul 26 18:42:01.838980 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:01.838945    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c42934d9-a5b1-4aff-9672-3d935f64f31f" path="/var/lib/kubelet/pods/c42934d9-a5b1-4aff-9672-3d935f64f31f/volumes"
Jul 26 18:42:04.174632 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:04.174592    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4147f2-drtjk"]
Jul 26 18:42:04.177423 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:04.177399    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4147f2-drtjk"]
Jul 26 18:42:05.838243 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:05.838200    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c8ad964-8934-426f-b9b0-20b7024ea6b2" path="/var/lib/kubelet/pods/9c8ad964-8934-426f-b9b0-20b7024ea6b2/volumes"
Jul 26 18:42:10.750604 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:10.750570    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"]
Jul 26 18:42:10.751045 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:10.750778    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m" podUID="33dfe2f4-147e-44a9-9d50-e7ac38670fd8" containerName="istio-proxy" containerID="cri-o://f5aa0de90dccf255abe3c92bb1ad1b412474a398afccd3ec148978b4080fd698" gracePeriod=30
Jul 26 18:42:14.290957 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:14.290913    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-76b936-fvqfd"]
Jul 26 18:42:14.291406 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:14.291344    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c8ad964-8934-426f-b9b0-20b7024ea6b2" containerName="revoke-keys"
Jul 26 18:42:14.291406 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:14.291356    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c8ad964-8934-426f-b9b0-20b7024ea6b2" containerName="revoke-keys"
Jul 26 18:42:14.291406 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:14.291378    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c42934d9-a5b1-4aff-9672-3d935f64f31f" containerName="maas-api"
Jul 26 18:42:14.291406 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:14.291384    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c42934d9-a5b1-4aff-9672-3d935f64f31f" containerName="maas-api"
Jul 26 18:42:14.291553 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:14.291445    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c42934d9-a5b1-4aff-9672-3d935f64f31f" containerName="maas-api"
Jul 26 18:42:14.291553 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:14.291453    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c8ad964-8934-426f-b9b0-20b7024ea6b2" containerName="revoke-keys"
Jul 26 18:42:14.296275 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:14.296258    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-76b936-fvqfd"
Jul 26 18:42:14.298649 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:14.298621    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-swzk5\""
Jul 26 18:42:14.301228 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:14.301202    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-76b936-fvqfd"]
Jul 26 18:42:14.391832 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:14.391793    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74d4cf24-8874-4304-9b56-36f8c1ae05d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-76b936-fvqfd\" (UID: \"74d4cf24-8874-4304-9b56-36f8c1ae05d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-76b936-fvqfd"
Jul 26 18:42:14.492945 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:14.492909    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74d4cf24-8874-4304-9b56-36f8c1ae05d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-76b936-fvqfd\" (UID: \"74d4cf24-8874-4304-9b56-36f8c1ae05d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-76b936-fvqfd"
Jul 26 18:42:14.493587 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:14.493567    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74d4cf24-8874-4304-9b56-36f8c1ae05d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-76b936-fvqfd\" (UID: \"74d4cf24-8874-4304-9b56-36f8c1ae05d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-76b936-fvqfd"
Jul 26 18:42:14.609652 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:14.609542    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-76b936-fvqfd"
Jul 26 18:42:14.741305 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:14.741277    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-76b936-fvqfd"]
Jul 26 18:42:14.743190 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:42:14.743160    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74d4cf24_8874_4304_9b56_36f8c1ae05d5.slice/crio-9981fec7285c1cccade9e37b64fcaf3a11fb8794143ad200655d95adf6b2804d WatchSource:0}: Error finding container 9981fec7285c1cccade9e37b64fcaf3a11fb8794143ad200655d95adf6b2804d: Status 404 returned error can't find the container with id 9981fec7285c1cccade9e37b64fcaf3a11fb8794143ad200655d95adf6b2804d
Jul 26 18:42:15.047178 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:15.047145    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-76b936-fvqfd" event={"ID":"74d4cf24-8874-4304-9b56-36f8c1ae05d5","Type":"ContainerStarted","Data":"9981fec7285c1cccade9e37b64fcaf3a11fb8794143ad200655d95adf6b2804d"}
Jul 26 18:42:16.013506 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.013477    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:42:16.055395 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.055356    2570 generic.go:356] "Generic (PLEG): container finished" podID="33dfe2f4-147e-44a9-9d50-e7ac38670fd8" containerID="f5aa0de90dccf255abe3c92bb1ad1b412474a398afccd3ec148978b4080fd698" exitCode=0
Jul 26 18:42:16.055586 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.055394    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m" event={"ID":"33dfe2f4-147e-44a9-9d50-e7ac38670fd8","Type":"ContainerDied","Data":"f5aa0de90dccf255abe3c92bb1ad1b412474a398afccd3ec148978b4080fd698"}
Jul 26 18:42:16.055586 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.055424    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"
Jul 26 18:42:16.055586 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.055456    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m" event={"ID":"33dfe2f4-147e-44a9-9d50-e7ac38670fd8","Type":"ContainerDied","Data":"7a4e31ea3537e43af2b5139c75ef8307538dfbd358f18d4f642bfc09a59892f0"}
Jul 26 18:42:16.055586 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.055487    2570 scope.go:117] "RemoveContainer" containerID="f5aa0de90dccf255abe3c92bb1ad1b412474a398afccd3ec148978b4080fd698"
Jul 26 18:42:16.057031 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.057004    2570 generic.go:356] "Generic (PLEG): container finished" podID="74d4cf24-8874-4304-9b56-36f8c1ae05d5" containerID="04e1188bf3939b3c03043d4c861eb7ce8b21d0f857a72a5ec9e246233f326712" exitCode=0
Jul 26 18:42:16.057152 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.057094    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-76b936-fvqfd" event={"ID":"74d4cf24-8874-4304-9b56-36f8c1ae05d5","Type":"ContainerDied","Data":"04e1188bf3939b3c03043d4c861eb7ce8b21d0f857a72a5ec9e246233f326712"}
Jul 26 18:42:16.065721 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.065694    2570 scope.go:117] "RemoveContainer" containerID="ccbc685793ffc30de7e4ae5c199d7d0458caca4d9bcce7c96e33e3f2c7932219"
Jul 26 18:42:16.075509 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.075491    2570 scope.go:117] "RemoveContainer" containerID="f5aa0de90dccf255abe3c92bb1ad1b412474a398afccd3ec148978b4080fd698"
Jul 26 18:42:16.075755 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:42:16.075737    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5aa0de90dccf255abe3c92bb1ad1b412474a398afccd3ec148978b4080fd698\": container with ID starting with f5aa0de90dccf255abe3c92bb1ad1b412474a398afccd3ec148978b4080fd698 not found: ID does not exist" containerID="f5aa0de90dccf255abe3c92bb1ad1b412474a398afccd3ec148978b4080fd698"
Jul 26 18:42:16.075813 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.075763    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5aa0de90dccf255abe3c92bb1ad1b412474a398afccd3ec148978b4080fd698"} err="failed to get container status \"f5aa0de90dccf255abe3c92bb1ad1b412474a398afccd3ec148978b4080fd698\": rpc error: code = NotFound desc = could not find container \"f5aa0de90dccf255abe3c92bb1ad1b412474a398afccd3ec148978b4080fd698\": container with ID starting with f5aa0de90dccf255abe3c92bb1ad1b412474a398afccd3ec148978b4080fd698 not found: ID does not exist"
Jul 26 18:42:16.075813 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.075779    2570 scope.go:117] "RemoveContainer" containerID="ccbc685793ffc30de7e4ae5c199d7d0458caca4d9bcce7c96e33e3f2c7932219"
Jul 26 18:42:16.076033 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:42:16.076013    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccbc685793ffc30de7e4ae5c199d7d0458caca4d9bcce7c96e33e3f2c7932219\": container with ID starting with ccbc685793ffc30de7e4ae5c199d7d0458caca4d9bcce7c96e33e3f2c7932219 not found: ID does not exist" containerID="ccbc685793ffc30de7e4ae5c199d7d0458caca4d9bcce7c96e33e3f2c7932219"
Jul 26 18:42:16.076076 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.076040    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccbc685793ffc30de7e4ae5c199d7d0458caca4d9bcce7c96e33e3f2c7932219"} err="failed to get container status \"ccbc685793ffc30de7e4ae5c199d7d0458caca4d9bcce7c96e33e3f2c7932219\": rpc error: code = NotFound desc = could not find container \"ccbc685793ffc30de7e4ae5c199d7d0458caca4d9bcce7c96e33e3f2c7932219\": container with ID starting with ccbc685793ffc30de7e4ae5c199d7d0458caca4d9bcce7c96e33e3f2c7932219 not found: ID does not exist"
Jul 26 18:42:16.108336 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.108310    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istiod-ca-cert\") pod \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") "
Jul 26 18:42:16.108456 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.108357    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-workload-certs\") pod \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") "
Jul 26 18:42:16.108456 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.108399    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istio-podinfo\") pod \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") "
Jul 26 18:42:16.108541 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.108497    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-credential-socket\") pod \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") "
Jul 26 18:42:16.108601 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.108555    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istio-data\") pod \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") "
Jul 26 18:42:16.108601 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.108593    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-workload-socket\") pod \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") "
Jul 26 18:42:16.108714 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.108640    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "33dfe2f4-147e-44a9-9d50-e7ac38670fd8" (UID: "33dfe2f4-147e-44a9-9d50-e7ac38670fd8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:42:16.108714 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.108658    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istio-envoy\") pod \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") "
Jul 26 18:42:16.108714 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.108690    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rb2x\" (UniqueName: \"kubernetes.io/projected/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-kube-api-access-5rb2x\") pod \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") "
Jul 26 18:42:16.108865 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.108716    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "33dfe2f4-147e-44a9-9d50-e7ac38670fd8" (UID: "33dfe2f4-147e-44a9-9d50-e7ac38670fd8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:42:16.108865 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.108729    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istio-token\") pod \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\" (UID: \"33dfe2f4-147e-44a9-9d50-e7ac38670fd8\") "
Jul 26 18:42:16.108865 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.108768    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "33dfe2f4-147e-44a9-9d50-e7ac38670fd8" (UID: "33dfe2f4-147e-44a9-9d50-e7ac38670fd8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:42:16.108992 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.108865    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "33dfe2f4-147e-44a9-9d50-e7ac38670fd8" (UID: "33dfe2f4-147e-44a9-9d50-e7ac38670fd8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:42:16.109087 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.109067    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-credential-socket\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:42:16.109167 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.109098    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-workload-socket\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:42:16.109167 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.109143    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istiod-ca-cert\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:42:16.109167 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.109158    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-workload-certs\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:42:16.109489 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.109462    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istio-data" (OuterVolumeSpecName: "istio-data") pod "33dfe2f4-147e-44a9-9d50-e7ac38670fd8" (UID: "33dfe2f4-147e-44a9-9d50-e7ac38670fd8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:42:16.110750 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.110722    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "33dfe2f4-147e-44a9-9d50-e7ac38670fd8" (UID: "33dfe2f4-147e-44a9-9d50-e7ac38670fd8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 26 18:42:16.110840 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.110750    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-kube-api-access-5rb2x" (OuterVolumeSpecName: "kube-api-access-5rb2x") pod "33dfe2f4-147e-44a9-9d50-e7ac38670fd8" (UID: "33dfe2f4-147e-44a9-9d50-e7ac38670fd8"). InnerVolumeSpecName "kube-api-access-5rb2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:42:16.110840 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.110821    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "33dfe2f4-147e-44a9-9d50-e7ac38670fd8" (UID: "33dfe2f4-147e-44a9-9d50-e7ac38670fd8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:42:16.110921 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.110846    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istio-token" (OuterVolumeSpecName: "istio-token") pod "33dfe2f4-147e-44a9-9d50-e7ac38670fd8" (UID: "33dfe2f4-147e-44a9-9d50-e7ac38670fd8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:42:16.209696 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.209601    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istio-token\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:42:16.209696 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.209638    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istio-podinfo\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:42:16.209696 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.209647    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istio-data\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:42:16.209696 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.209655    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-istio-envoy\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:42:16.209696 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.209663    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rb2x\" (UniqueName: \"kubernetes.io/projected/33dfe2f4-147e-44a9-9d50-e7ac38670fd8-kube-api-access-5rb2x\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:42:16.377369 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.377332    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"]
Jul 26 18:42:16.384249 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:16.384218    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4147f2-openshift-default-764fdfcc48-9gv2m"]
Jul 26 18:42:17.204410 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:17.204380    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-76b936-fvqfd"
Jul 26 18:42:17.319564 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:17.319464    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74d4cf24-8874-4304-9b56-36f8c1ae05d5-maas-api-service-ca\") pod \"74d4cf24-8874-4304-9b56-36f8c1ae05d5\" (UID: \"74d4cf24-8874-4304-9b56-36f8c1ae05d5\") "
Jul 26 18:42:17.319850 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:17.319827    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74d4cf24-8874-4304-9b56-36f8c1ae05d5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "74d4cf24-8874-4304-9b56-36f8c1ae05d5" (UID: "74d4cf24-8874-4304-9b56-36f8c1ae05d5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:42:17.420492 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:17.420455    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74d4cf24-8874-4304-9b56-36f8c1ae05d5-maas-api-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:42:17.845932 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:17.845894    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33dfe2f4-147e-44a9-9d50-e7ac38670fd8" path="/var/lib/kubelet/pods/33dfe2f4-147e-44a9-9d50-e7ac38670fd8/volumes"
Jul 26 18:42:18.071361 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:18.071332    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-76b936-fvqfd"
Jul 26 18:42:18.071361 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:18.071347    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-76b936-fvqfd" event={"ID":"74d4cf24-8874-4304-9b56-36f8c1ae05d5","Type":"ContainerDied","Data":"9981fec7285c1cccade9e37b64fcaf3a11fb8794143ad200655d95adf6b2804d"}
Jul 26 18:42:18.071562 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:18.071394    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9981fec7285c1cccade9e37b64fcaf3a11fb8794143ad200655d95adf6b2804d"
Jul 26 18:42:29.770147 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:29.770093    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk"]
Jul 26 18:42:29.770768 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:29.770387    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk" podUID="806fa5fa-a85e-4d5c-9599-67e49291e2f3" containerName="maas-api" containerID="cri-o://e22c66fd6a7108006687dfce20b1b027cf746cb7e3ff9274f1900414f30ce9f4" gracePeriod=30
Jul 26 18:42:30.029655 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:30.029581    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk"
Jul 26 18:42:30.032859 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:30.032835    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mt84b\" (UniqueName: \"kubernetes.io/projected/806fa5fa-a85e-4d5c-9599-67e49291e2f3-kube-api-access-mt84b\") pod \"806fa5fa-a85e-4d5c-9599-67e49291e2f3\" (UID: \"806fa5fa-a85e-4d5c-9599-67e49291e2f3\") "
Jul 26 18:42:30.033007 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:30.032880    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/806fa5fa-a85e-4d5c-9599-67e49291e2f3-openshift-service-ca\") pod \"806fa5fa-a85e-4d5c-9599-67e49291e2f3\" (UID: \"806fa5fa-a85e-4d5c-9599-67e49291e2f3\") "
Jul 26 18:42:30.033007 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:30.032965    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/806fa5fa-a85e-4d5c-9599-67e49291e2f3-maas-api-tls\") pod \"806fa5fa-a85e-4d5c-9599-67e49291e2f3\" (UID: \"806fa5fa-a85e-4d5c-9599-67e49291e2f3\") "
Jul 26 18:42:30.033346 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:30.033323    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/806fa5fa-a85e-4d5c-9599-67e49291e2f3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "806fa5fa-a85e-4d5c-9599-67e49291e2f3" (UID: "806fa5fa-a85e-4d5c-9599-67e49291e2f3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:42:30.035013 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:30.034990    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/806fa5fa-a85e-4d5c-9599-67e49291e2f3-kube-api-access-mt84b" (OuterVolumeSpecName: "kube-api-access-mt84b") pod "806fa5fa-a85e-4d5c-9599-67e49291e2f3" (UID: "806fa5fa-a85e-4d5c-9599-67e49291e2f3"). InnerVolumeSpecName "kube-api-access-mt84b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:42:30.035013 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:30.034999    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/806fa5fa-a85e-4d5c-9599-67e49291e2f3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "806fa5fa-a85e-4d5c-9599-67e49291e2f3" (UID: "806fa5fa-a85e-4d5c-9599-67e49291e2f3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:42:30.134137 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:30.134084    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/806fa5fa-a85e-4d5c-9599-67e49291e2f3-maas-api-tls\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:42:30.134137 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:30.134124    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mt84b\" (UniqueName: \"kubernetes.io/projected/806fa5fa-a85e-4d5c-9599-67e49291e2f3-kube-api-access-mt84b\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:42:30.134137 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:30.134138    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/806fa5fa-a85e-4d5c-9599-67e49291e2f3-openshift-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:42:30.149149 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:30.149087    2570 generic.go:356] "Generic (PLEG): container finished" podID="806fa5fa-a85e-4d5c-9599-67e49291e2f3" containerID="e22c66fd6a7108006687dfce20b1b027cf746cb7e3ff9274f1900414f30ce9f4" exitCode=0
Jul 26 18:42:30.149287 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:30.149156    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk" event={"ID":"806fa5fa-a85e-4d5c-9599-67e49291e2f3","Type":"ContainerDied","Data":"e22c66fd6a7108006687dfce20b1b027cf746cb7e3ff9274f1900414f30ce9f4"}
Jul 26 18:42:30.149287 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:30.149204    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk" event={"ID":"806fa5fa-a85e-4d5c-9599-67e49291e2f3","Type":"ContainerDied","Data":"a2bc0ae8bf58200cbda349d061e12f594f453f8d9c966e00cad55236edeae25f"}
Jul 26 18:42:30.149287 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:30.149226    2570 scope.go:117] "RemoveContainer" containerID="e22c66fd6a7108006687dfce20b1b027cf746cb7e3ff9274f1900414f30ce9f4"
Jul 26 18:42:30.149287 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:30.149161    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk"
Jul 26 18:42:30.159494 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:30.159471    2570 scope.go:117] "RemoveContainer" containerID="e22c66fd6a7108006687dfce20b1b027cf746cb7e3ff9274f1900414f30ce9f4"
Jul 26 18:42:30.159730 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:42:30.159709    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e22c66fd6a7108006687dfce20b1b027cf746cb7e3ff9274f1900414f30ce9f4\": container with ID starting with e22c66fd6a7108006687dfce20b1b027cf746cb7e3ff9274f1900414f30ce9f4 not found: ID does not exist" containerID="e22c66fd6a7108006687dfce20b1b027cf746cb7e3ff9274f1900414f30ce9f4"
Jul 26 18:42:30.159785 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:30.159742    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e22c66fd6a7108006687dfce20b1b027cf746cb7e3ff9274f1900414f30ce9f4"} err="failed to get container status \"e22c66fd6a7108006687dfce20b1b027cf746cb7e3ff9274f1900414f30ce9f4\": rpc error: code = NotFound desc = could not find container \"e22c66fd6a7108006687dfce20b1b027cf746cb7e3ff9274f1900414f30ce9f4\": container with ID starting with e22c66fd6a7108006687dfce20b1b027cf746cb7e3ff9274f1900414f30ce9f4 not found: ID does not exist"
Jul 26 18:42:30.172876 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:30.172850    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk"]
Jul 26 18:42:30.177199 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:30.177175    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-76b936-768f675c59-77wgk"]
Jul 26 18:42:31.838630 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:31.838598    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="806fa5fa-a85e-4d5c-9599-67e49291e2f3" path="/var/lib/kubelet/pods/806fa5fa-a85e-4d5c-9599-67e49291e2f3/volumes"
Jul 26 18:42:34.362813 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:34.362774    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-76b936-fvqfd"]
Jul 26 18:42:34.365053 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:34.365029    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-76b936-fvqfd"]
Jul 26 18:42:35.839284 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:35.839249    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74d4cf24-8874-4304-9b56-36f8c1ae05d5" path="/var/lib/kubelet/pods/74d4cf24-8874-4304-9b56-36f8c1ae05d5/volumes"
Jul 26 18:42:39.744387 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:39.744351    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"]
Jul 26 18:42:54.763871 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:54.763833    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7" podUID="07bb2b32-5d3c-47b7-9998-f71e03c5a17e" containerName="main" containerID="cri-o://444b8f3c73a1403fe7e489b25a3cbf3b12494ade70c88bae4d6dc33d465e5946" gracePeriod=45
Jul 26 18:42:55.012955 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.012929    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:42:55.150655 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.150547    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-tls-certs\") pod \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\" (UID: \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\") "
Jul 26 18:42:55.150655 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.150607    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ft567\" (UniqueName: \"kubernetes.io/projected/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-kube-api-access-ft567\") pod \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\" (UID: \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\") "
Jul 26 18:42:55.150655 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.150645    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-dshm\") pod \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\" (UID: \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\") "
Jul 26 18:42:55.150968 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.150681    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-home\") pod \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\" (UID: \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\") "
Jul 26 18:42:55.150968 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.150703    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-model-cache\") pod \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\" (UID: \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\") "
Jul 26 18:42:55.150968 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.150750    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-tmp-dir\") pod \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\" (UID: \"07bb2b32-5d3c-47b7-9998-f71e03c5a17e\") "
Jul 26 18:42:55.151154 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.151014    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-home" (OuterVolumeSpecName: "home") pod "07bb2b32-5d3c-47b7-9998-f71e03c5a17e" (UID: "07bb2b32-5d3c-47b7-9998-f71e03c5a17e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:42:55.151154 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.151030    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-model-cache" (OuterVolumeSpecName: "model-cache") pod "07bb2b32-5d3c-47b7-9998-f71e03c5a17e" (UID: "07bb2b32-5d3c-47b7-9998-f71e03c5a17e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:42:55.151154 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.151042    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "07bb2b32-5d3c-47b7-9998-f71e03c5a17e" (UID: "07bb2b32-5d3c-47b7-9998-f71e03c5a17e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:42:55.153052 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.153019    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-kube-api-access-ft567" (OuterVolumeSpecName: "kube-api-access-ft567") pod "07bb2b32-5d3c-47b7-9998-f71e03c5a17e" (UID: "07bb2b32-5d3c-47b7-9998-f71e03c5a17e"). InnerVolumeSpecName "kube-api-access-ft567". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:42:55.153052 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.153043    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "07bb2b32-5d3c-47b7-9998-f71e03c5a17e" (UID: "07bb2b32-5d3c-47b7-9998-f71e03c5a17e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:42:55.153229 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.153059    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-dshm" (OuterVolumeSpecName: "dshm") pod "07bb2b32-5d3c-47b7-9998-f71e03c5a17e" (UID: "07bb2b32-5d3c-47b7-9998-f71e03c5a17e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:42:55.252223 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.252182    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-home\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:42:55.252223 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.252218    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-model-cache\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:42:55.252223 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.252233    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-tmp-dir\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:42:55.252468 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.252245    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-tls-certs\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:42:55.252468 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.252257    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ft567\" (UniqueName: \"kubernetes.io/projected/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-kube-api-access-ft567\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:42:55.252468 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.252269    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07bb2b32-5d3c-47b7-9998-f71e03c5a17e-dshm\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:42:55.294289 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.294253    2570 generic.go:356] "Generic (PLEG): container finished" podID="07bb2b32-5d3c-47b7-9998-f71e03c5a17e" containerID="444b8f3c73a1403fe7e489b25a3cbf3b12494ade70c88bae4d6dc33d465e5946" exitCode=0
Jul 26 18:42:55.294448 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.294293    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"
Jul 26 18:42:55.294448 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.294336    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7" event={"ID":"07bb2b32-5d3c-47b7-9998-f71e03c5a17e","Type":"ContainerDied","Data":"444b8f3c73a1403fe7e489b25a3cbf3b12494ade70c88bae4d6dc33d465e5946"}
Jul 26 18:42:55.294448 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.294376    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7" event={"ID":"07bb2b32-5d3c-47b7-9998-f71e03c5a17e","Type":"ContainerDied","Data":"961382d6b6634b879a840ddf7ca0a002c3b49e538aaf8aa005bc8a14d0c95246"}
Jul 26 18:42:55.294448 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.294394    2570 scope.go:117] "RemoveContainer" containerID="444b8f3c73a1403fe7e489b25a3cbf3b12494ade70c88bae4d6dc33d465e5946"
Jul 26 18:42:55.304641 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.304599    2570 scope.go:117] "RemoveContainer" containerID="444b8f3c73a1403fe7e489b25a3cbf3b12494ade70c88bae4d6dc33d465e5946"
Jul 26 18:42:55.304879 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:42:55.304862    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"444b8f3c73a1403fe7e489b25a3cbf3b12494ade70c88bae4d6dc33d465e5946\": container with ID starting with 444b8f3c73a1403fe7e489b25a3cbf3b12494ade70c88bae4d6dc33d465e5946 not found: ID does not exist" containerID="444b8f3c73a1403fe7e489b25a3cbf3b12494ade70c88bae4d6dc33d465e5946"
Jul 26 18:42:55.304948 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.304886    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"444b8f3c73a1403fe7e489b25a3cbf3b12494ade70c88bae4d6dc33d465e5946"} err="failed to get container status \"444b8f3c73a1403fe7e489b25a3cbf3b12494ade70c88bae4d6dc33d465e5946\": rpc error: code = NotFound desc = could not find container \"444b8f3c73a1403fe7e489b25a3cbf3b12494ade70c88bae4d6dc33d465e5946\": container with ID starting with 444b8f3c73a1403fe7e489b25a3cbf3b12494ade70c88bae4d6dc33d465e5946 not found: ID does not exist"
Jul 26 18:42:55.316197 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.316175    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"]
Jul 26 18:42:55.319917 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.319895    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-76b936/ipp-route-76b936-kserve-787664fcbc-tjvb7"]
Jul 26 18:42:55.838438 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:42:55.838406    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07bb2b32-5d3c-47b7-9998-f71e03c5a17e" path="/var/lib/kubelet/pods/07bb2b32-5d3c-47b7-9998-f71e03c5a17e/volumes"
Jul 26 18:43:22.905359 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:22.905324    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e8329-vg7hv"]
Jul 26 18:43:22.905814 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:22.905680    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07bb2b32-5d3c-47b7-9998-f71e03c5a17e" containerName="main"
Jul 26 18:43:22.905814 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:22.905692    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="07bb2b32-5d3c-47b7-9998-f71e03c5a17e" containerName="main"
Jul 26 18:43:22.905814 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:22.905700    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33dfe2f4-147e-44a9-9d50-e7ac38670fd8" containerName="istio-proxy"
Jul 26 18:43:22.905814 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:22.905705    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="33dfe2f4-147e-44a9-9d50-e7ac38670fd8" containerName="istio-proxy"
Jul 26 18:43:22.905814 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:22.905720    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33dfe2f4-147e-44a9-9d50-e7ac38670fd8" containerName="istio-proxy"
Jul 26 18:43:22.905814 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:22.905725    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="33dfe2f4-147e-44a9-9d50-e7ac38670fd8" containerName="istio-proxy"
Jul 26 18:43:22.905814 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:22.905735    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="806fa5fa-a85e-4d5c-9599-67e49291e2f3" containerName="maas-api"
Jul 26 18:43:22.905814 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:22.905740    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="806fa5fa-a85e-4d5c-9599-67e49291e2f3" containerName="maas-api"
Jul 26 18:43:22.905814 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:22.905751    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74d4cf24-8874-4304-9b56-36f8c1ae05d5" containerName="revoke-keys"
Jul 26 18:43:22.905814 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:22.905756    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="74d4cf24-8874-4304-9b56-36f8c1ae05d5" containerName="revoke-keys"
Jul 26 18:43:22.905814 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:22.905816    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="07bb2b32-5d3c-47b7-9998-f71e03c5a17e" containerName="main"
Jul 26 18:43:22.906186 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:22.905826    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="33dfe2f4-147e-44a9-9d50-e7ac38670fd8" containerName="istio-proxy"
Jul 26 18:43:22.906186 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:22.905831    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="74d4cf24-8874-4304-9b56-36f8c1ae05d5" containerName="revoke-keys"
Jul 26 18:43:22.906186 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:22.905838    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="806fa5fa-a85e-4d5c-9599-67e49291e2f3" containerName="maas-api"
Jul 26 18:43:22.909004 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:22.908988    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e8329-vg7hv"
Jul 26 18:43:22.911152 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:22.911132    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-swzk5\""
Jul 26 18:43:22.915264 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:22.915238    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e8329-vg7hv"]
Jul 26 18:43:23.004459 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:23.004427    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/783bd2ef-8ee4-48eb-92ca-17f9eb327229-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-3e8329-vg7hv\" (UID: \"783bd2ef-8ee4-48eb-92ca-17f9eb327229\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e8329-vg7hv"
Jul 26 18:43:23.105548 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:23.105511    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/783bd2ef-8ee4-48eb-92ca-17f9eb327229-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-3e8329-vg7hv\" (UID: \"783bd2ef-8ee4-48eb-92ca-17f9eb327229\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e8329-vg7hv"
Jul 26 18:43:23.106162 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:23.106142    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/783bd2ef-8ee4-48eb-92ca-17f9eb327229-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-3e8329-vg7hv\" (UID: \"783bd2ef-8ee4-48eb-92ca-17f9eb327229\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e8329-vg7hv"
Jul 26 18:43:23.220885 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:23.220783    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e8329-vg7hv"
Jul 26 18:43:23.357680 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:23.357648    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e8329-vg7hv"]
Jul 26 18:43:23.359502 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:43:23.359478    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod783bd2ef_8ee4_48eb_92ca_17f9eb327229.slice/crio-35516ec0058b2e266bbed756587cafff0e3f5d8fc33b8ea51ccb23ac529566c2 WatchSource:0}: Error finding container 35516ec0058b2e266bbed756587cafff0e3f5d8fc33b8ea51ccb23ac529566c2: Status 404 returned error can't find the container with id 35516ec0058b2e266bbed756587cafff0e3f5d8fc33b8ea51ccb23ac529566c2
Jul 26 18:43:23.456072 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:23.456031    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e8329-vg7hv" event={"ID":"783bd2ef-8ee4-48eb-92ca-17f9eb327229","Type":"ContainerStarted","Data":"35516ec0058b2e266bbed756587cafff0e3f5d8fc33b8ea51ccb23ac529566c2"}
Jul 26 18:43:24.463485 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:24.463394    2570 generic.go:356] "Generic (PLEG): container finished" podID="783bd2ef-8ee4-48eb-92ca-17f9eb327229" containerID="100fc9b5d70d49166aa4fe6e891f8a50ff32edc29f0c20d6bbead0e3492281b8" exitCode=0
Jul 26 18:43:24.463840 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:24.463486    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e8329-vg7hv" event={"ID":"783bd2ef-8ee4-48eb-92ca-17f9eb327229","Type":"ContainerDied","Data":"100fc9b5d70d49166aa4fe6e891f8a50ff32edc29f0c20d6bbead0e3492281b8"}
Jul 26 18:43:25.603739 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:25.603715    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e8329-vg7hv"
Jul 26 18:43:25.628129 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:25.628082    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/783bd2ef-8ee4-48eb-92ca-17f9eb327229-maas-api-service-ca\") pod \"783bd2ef-8ee4-48eb-92ca-17f9eb327229\" (UID: \"783bd2ef-8ee4-48eb-92ca-17f9eb327229\") "
Jul 26 18:43:25.628410 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:25.628388    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/783bd2ef-8ee4-48eb-92ca-17f9eb327229-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "783bd2ef-8ee4-48eb-92ca-17f9eb327229" (UID: "783bd2ef-8ee4-48eb-92ca-17f9eb327229"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:43:25.729410 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:25.729327    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/783bd2ef-8ee4-48eb-92ca-17f9eb327229-maas-api-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:43:26.477034 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:26.476985    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e8329-vg7hv" event={"ID":"783bd2ef-8ee4-48eb-92ca-17f9eb327229","Type":"ContainerDied","Data":"35516ec0058b2e266bbed756587cafff0e3f5d8fc33b8ea51ccb23ac529566c2"}
Jul 26 18:43:26.477259 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:26.477124    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="35516ec0058b2e266bbed756587cafff0e3f5d8fc33b8ea51ccb23ac529566c2"
Jul 26 18:43:26.477259 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:26.477210    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e8329-vg7hv"
Jul 26 18:43:33.460504 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:33.460466    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82"]
Jul 26 18:43:33.461092 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:33.460714    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82" podUID="005b8b98-b1f3-494a-887a-40ec533c1a46" containerName="maas-api" containerID="cri-o://42da71bd7f4e73d1d1b73810b57cbf27ff1971b259e9a2eda3b9c18e6e1f8f09" gracePeriod=30
Jul 26 18:43:33.712659 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:33.712597    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82"
Jul 26 18:43:33.801281 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:33.801245    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/005b8b98-b1f3-494a-887a-40ec533c1a46-openshift-service-ca\") pod \"005b8b98-b1f3-494a-887a-40ec533c1a46\" (UID: \"005b8b98-b1f3-494a-887a-40ec533c1a46\") "
Jul 26 18:43:33.801281 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:33.801282    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/005b8b98-b1f3-494a-887a-40ec533c1a46-maas-api-tls\") pod \"005b8b98-b1f3-494a-887a-40ec533c1a46\" (UID: \"005b8b98-b1f3-494a-887a-40ec533c1a46\") "
Jul 26 18:43:33.801522 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:33.801365    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fn7jl\" (UniqueName: \"kubernetes.io/projected/005b8b98-b1f3-494a-887a-40ec533c1a46-kube-api-access-fn7jl\") pod \"005b8b98-b1f3-494a-887a-40ec533c1a46\" (UID: \"005b8b98-b1f3-494a-887a-40ec533c1a46\") "
Jul 26 18:43:33.801685 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:33.801657    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/005b8b98-b1f3-494a-887a-40ec533c1a46-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "005b8b98-b1f3-494a-887a-40ec533c1a46" (UID: "005b8b98-b1f3-494a-887a-40ec533c1a46"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:43:33.803476 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:33.803449    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/005b8b98-b1f3-494a-887a-40ec533c1a46-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "005b8b98-b1f3-494a-887a-40ec533c1a46" (UID: "005b8b98-b1f3-494a-887a-40ec533c1a46"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:43:33.803588 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:33.803523    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/005b8b98-b1f3-494a-887a-40ec533c1a46-kube-api-access-fn7jl" (OuterVolumeSpecName: "kube-api-access-fn7jl") pod "005b8b98-b1f3-494a-887a-40ec533c1a46" (UID: "005b8b98-b1f3-494a-887a-40ec533c1a46"). InnerVolumeSpecName "kube-api-access-fn7jl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:43:33.902995 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:33.902951    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/005b8b98-b1f3-494a-887a-40ec533c1a46-openshift-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:43:33.902995 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:33.902988    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/005b8b98-b1f3-494a-887a-40ec533c1a46-maas-api-tls\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:43:33.902995 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:33.903001    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fn7jl\" (UniqueName: \"kubernetes.io/projected/005b8b98-b1f3-494a-887a-40ec533c1a46-kube-api-access-fn7jl\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:43:34.525067 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:34.525032    2570 generic.go:356] "Generic (PLEG): container finished" podID="005b8b98-b1f3-494a-887a-40ec533c1a46" containerID="42da71bd7f4e73d1d1b73810b57cbf27ff1971b259e9a2eda3b9c18e6e1f8f09" exitCode=0
Jul 26 18:43:34.525603 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:34.525084    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82"
Jul 26 18:43:34.525603 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:34.525133    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82" event={"ID":"005b8b98-b1f3-494a-887a-40ec533c1a46","Type":"ContainerDied","Data":"42da71bd7f4e73d1d1b73810b57cbf27ff1971b259e9a2eda3b9c18e6e1f8f09"}
Jul 26 18:43:34.525603 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:34.525194    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82" event={"ID":"005b8b98-b1f3-494a-887a-40ec533c1a46","Type":"ContainerDied","Data":"2a78333849805c4bee4a907912bdea26cdaf63697a2e7b997422d269404a5900"}
Jul 26 18:43:34.525603 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:34.525230    2570 scope.go:117] "RemoveContainer" containerID="42da71bd7f4e73d1d1b73810b57cbf27ff1971b259e9a2eda3b9c18e6e1f8f09"
Jul 26 18:43:34.534503 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:34.534484    2570 scope.go:117] "RemoveContainer" containerID="42da71bd7f4e73d1d1b73810b57cbf27ff1971b259e9a2eda3b9c18e6e1f8f09"
Jul 26 18:43:34.534748 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:43:34.534731    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42da71bd7f4e73d1d1b73810b57cbf27ff1971b259e9a2eda3b9c18e6e1f8f09\": container with ID starting with 42da71bd7f4e73d1d1b73810b57cbf27ff1971b259e9a2eda3b9c18e6e1f8f09 not found: ID does not exist" containerID="42da71bd7f4e73d1d1b73810b57cbf27ff1971b259e9a2eda3b9c18e6e1f8f09"
Jul 26 18:43:34.534799 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:34.534757    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42da71bd7f4e73d1d1b73810b57cbf27ff1971b259e9a2eda3b9c18e6e1f8f09"} err="failed to get container status \"42da71bd7f4e73d1d1b73810b57cbf27ff1971b259e9a2eda3b9c18e6e1f8f09\": rpc error: code = NotFound desc = could not find container \"42da71bd7f4e73d1d1b73810b57cbf27ff1971b259e9a2eda3b9c18e6e1f8f09\": container with ID starting with 42da71bd7f4e73d1d1b73810b57cbf27ff1971b259e9a2eda3b9c18e6e1f8f09 not found: ID does not exist"
Jul 26 18:43:34.541220 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:34.541199    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82"]
Jul 26 18:43:34.544599 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:34.544578    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-3e8329-6d6c7657d9-zcb82"]
Jul 26 18:43:35.838465 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:35.838431    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="005b8b98-b1f3-494a-887a-40ec533c1a46" path="/var/lib/kubelet/pods/005b8b98-b1f3-494a-887a-40ec533c1a46/volumes"
Jul 26 18:43:37.975588 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:37.975555    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e8329-vg7hv"]
Jul 26 18:43:37.978523 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:37.978496    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e8329-vg7hv"]
Jul 26 18:43:39.838355 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:39.838308    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="783bd2ef-8ee4-48eb-92ca-17f9eb327229" path="/var/lib/kubelet/pods/783bd2ef-8ee4-48eb-92ca-17f9eb327229/volumes"
Jul 26 18:43:44.673583 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:44.673546    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-57f93c-9tqj8"]
Jul 26 18:43:44.673953 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:44.673910    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="783bd2ef-8ee4-48eb-92ca-17f9eb327229" containerName="revoke-keys"
Jul 26 18:43:44.673953 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:44.673921    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="783bd2ef-8ee4-48eb-92ca-17f9eb327229" containerName="revoke-keys"
Jul 26 18:43:44.673953 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:44.673942    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="005b8b98-b1f3-494a-887a-40ec533c1a46" containerName="maas-api"
Jul 26 18:43:44.673953 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:44.673948    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="005b8b98-b1f3-494a-887a-40ec533c1a46" containerName="maas-api"
Jul 26 18:43:44.674106 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:44.674005    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="005b8b98-b1f3-494a-887a-40ec533c1a46" containerName="maas-api"
Jul 26 18:43:44.674106 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:44.674032    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="783bd2ef-8ee4-48eb-92ca-17f9eb327229" containerName="revoke-keys"
Jul 26 18:43:44.674106 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:44.674040    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="33dfe2f4-147e-44a9-9d50-e7ac38670fd8" containerName="istio-proxy"
Jul 26 18:43:44.678554 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:44.678536    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-57f93c-9tqj8"
Jul 26 18:43:44.680684 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:44.680659    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-swzk5\""
Jul 26 18:43:44.687648 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:44.687619    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-57f93c-9tqj8"]
Jul 26 18:43:44.799813 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:44.799764    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a861acd7-d8cb-4378-9fdf-019ae929eb87-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-57f93c-9tqj8\" (UID: \"a861acd7-d8cb-4378-9fdf-019ae929eb87\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-57f93c-9tqj8"
Jul 26 18:43:44.900543 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:44.900512    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a861acd7-d8cb-4378-9fdf-019ae929eb87-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-57f93c-9tqj8\" (UID: \"a861acd7-d8cb-4378-9fdf-019ae929eb87\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-57f93c-9tqj8"
Jul 26 18:43:44.901204 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:44.901183    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a861acd7-d8cb-4378-9fdf-019ae929eb87-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-57f93c-9tqj8\" (UID: \"a861acd7-d8cb-4378-9fdf-019ae929eb87\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-57f93c-9tqj8"
Jul 26 18:43:45.000388 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:45.000301    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-57f93c-9tqj8"
Jul 26 18:43:45.137677 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:45.137651    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-57f93c-9tqj8"]
Jul 26 18:43:45.139195 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:43:45.139166    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda861acd7_d8cb_4378_9fdf_019ae929eb87.slice/crio-fd8a0c9fe05857fc1a75051c702e2ce6f4ebcd8659b0cfa222733b085c99f50c WatchSource:0}: Error finding container fd8a0c9fe05857fc1a75051c702e2ce6f4ebcd8659b0cfa222733b085c99f50c: Status 404 returned error can't find the container with id fd8a0c9fe05857fc1a75051c702e2ce6f4ebcd8659b0cfa222733b085c99f50c
Jul 26 18:43:45.589351 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:45.589309    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-57f93c-9tqj8" event={"ID":"a861acd7-d8cb-4378-9fdf-019ae929eb87","Type":"ContainerStarted","Data":"bcf827b6b34155db43a37eeba0c5529366e589f00cac5d409a2a8ec4c9073a27"}
Jul 26 18:43:45.589530 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:45.589374    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-57f93c-9tqj8" event={"ID":"a861acd7-d8cb-4378-9fdf-019ae929eb87","Type":"ContainerStarted","Data":"fd8a0c9fe05857fc1a75051c702e2ce6f4ebcd8659b0cfa222733b085c99f50c"}
Jul 26 18:43:45.604825 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:45.604713    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-57f93c-9tqj8" podStartSLOduration=1.6046961899999999 podStartE2EDuration="1.60469619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:43:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:43:45.603627032 +0000 UTC m=+3282.255817890" watchObservedRunningTime="2026-07-26 18:43:45.60469619 +0000 UTC m=+3282.256887045"
Jul 26 18:43:46.597464 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:46.597429    2570 generic.go:356] "Generic (PLEG): container finished" podID="a861acd7-d8cb-4378-9fdf-019ae929eb87" containerID="bcf827b6b34155db43a37eeba0c5529366e589f00cac5d409a2a8ec4c9073a27" exitCode=0
Jul 26 18:43:46.597856 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:46.597514    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-57f93c-9tqj8" event={"ID":"a861acd7-d8cb-4378-9fdf-019ae929eb87","Type":"ContainerDied","Data":"bcf827b6b34155db43a37eeba0c5529366e589f00cac5d409a2a8ec4c9073a27"}
Jul 26 18:43:47.741134 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:47.741091    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-57f93c-9tqj8"
Jul 26 18:43:47.830001 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:47.829974    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a861acd7-d8cb-4378-9fdf-019ae929eb87-maas-api-service-ca\") pod \"a861acd7-d8cb-4378-9fdf-019ae929eb87\" (UID: \"a861acd7-d8cb-4378-9fdf-019ae929eb87\") "
Jul 26 18:43:47.830357 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:47.830333    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a861acd7-d8cb-4378-9fdf-019ae929eb87-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a861acd7-d8cb-4378-9fdf-019ae929eb87" (UID: "a861acd7-d8cb-4378-9fdf-019ae929eb87"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:43:47.931480 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:47.931387    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a861acd7-d8cb-4378-9fdf-019ae929eb87-maas-api-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:43:48.614522 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:48.614488    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-57f93c-9tqj8"
Jul 26 18:43:48.614522 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:48.614503    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-57f93c-9tqj8" event={"ID":"a861acd7-d8cb-4378-9fdf-019ae929eb87","Type":"ContainerDied","Data":"fd8a0c9fe05857fc1a75051c702e2ce6f4ebcd8659b0cfa222733b085c99f50c"}
Jul 26 18:43:48.614741 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:48.614549    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fd8a0c9fe05857fc1a75051c702e2ce6f4ebcd8659b0cfa222733b085c99f50c"
Jul 26 18:43:55.151154 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.151103    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9"]
Jul 26 18:43:55.151665 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.151329    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9" podUID="ff60c926-d113-4a1a-ac00-faa9791b6cdd" containerName="maas-api" containerID="cri-o://e8369c4fbb455a3e6abce790821ea9d3d1668210d7c57ea78161cc822586e57c" gracePeriod=30
Jul 26 18:43:55.193860 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.193825    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-57f93c-29751510-rpcqg"]
Jul 26 18:43:55.198129 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.198084    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-57f93c-29751510-rpcqg"]
Jul 26 18:43:55.409034 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.408969    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9"
Jul 26 18:43:55.499400 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.499363    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58cxw\" (UniqueName: \"kubernetes.io/projected/ff60c926-d113-4a1a-ac00-faa9791b6cdd-kube-api-access-58cxw\") pod \"ff60c926-d113-4a1a-ac00-faa9791b6cdd\" (UID: \"ff60c926-d113-4a1a-ac00-faa9791b6cdd\") "
Jul 26 18:43:55.499591 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.499416    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff60c926-d113-4a1a-ac00-faa9791b6cdd-openshift-service-ca\") pod \"ff60c926-d113-4a1a-ac00-faa9791b6cdd\" (UID: \"ff60c926-d113-4a1a-ac00-faa9791b6cdd\") "
Jul 26 18:43:55.499591 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.499456    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff60c926-d113-4a1a-ac00-faa9791b6cdd-maas-api-tls\") pod \"ff60c926-d113-4a1a-ac00-faa9791b6cdd\" (UID: \"ff60c926-d113-4a1a-ac00-faa9791b6cdd\") "
Jul 26 18:43:55.499824 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.499784    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff60c926-d113-4a1a-ac00-faa9791b6cdd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ff60c926-d113-4a1a-ac00-faa9791b6cdd" (UID: "ff60c926-d113-4a1a-ac00-faa9791b6cdd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:43:55.501532 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.501511    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff60c926-d113-4a1a-ac00-faa9791b6cdd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ff60c926-d113-4a1a-ac00-faa9791b6cdd" (UID: "ff60c926-d113-4a1a-ac00-faa9791b6cdd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:43:55.501651 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.501632    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff60c926-d113-4a1a-ac00-faa9791b6cdd-kube-api-access-58cxw" (OuterVolumeSpecName: "kube-api-access-58cxw") pod "ff60c926-d113-4a1a-ac00-faa9791b6cdd" (UID: "ff60c926-d113-4a1a-ac00-faa9791b6cdd"). InnerVolumeSpecName "kube-api-access-58cxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:43:55.600990 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.600956    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff60c926-d113-4a1a-ac00-faa9791b6cdd-maas-api-tls\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:43:55.600990 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.600983    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58cxw\" (UniqueName: \"kubernetes.io/projected/ff60c926-d113-4a1a-ac00-faa9791b6cdd-kube-api-access-58cxw\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:43:55.600990 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.600994    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff60c926-d113-4a1a-ac00-faa9791b6cdd-openshift-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:43:55.656389 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.656351    2570 generic.go:356] "Generic (PLEG): container finished" podID="ff60c926-d113-4a1a-ac00-faa9791b6cdd" containerID="e8369c4fbb455a3e6abce790821ea9d3d1668210d7c57ea78161cc822586e57c" exitCode=0
Jul 26 18:43:55.656572 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.656401    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9"
Jul 26 18:43:55.656572 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.656426    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9" event={"ID":"ff60c926-d113-4a1a-ac00-faa9791b6cdd","Type":"ContainerDied","Data":"e8369c4fbb455a3e6abce790821ea9d3d1668210d7c57ea78161cc822586e57c"}
Jul 26 18:43:55.656572 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.656473    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9" event={"ID":"ff60c926-d113-4a1a-ac00-faa9791b6cdd","Type":"ContainerDied","Data":"c8d15cdcf96ebd3fcb8b61af56b2f62a730c2bede20774c7dd614c7d821cf07b"}
Jul 26 18:43:55.656572 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.656498    2570 scope.go:117] "RemoveContainer" containerID="e8369c4fbb455a3e6abce790821ea9d3d1668210d7c57ea78161cc822586e57c"
Jul 26 18:43:55.666360 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.666339    2570 scope.go:117] "RemoveContainer" containerID="e8369c4fbb455a3e6abce790821ea9d3d1668210d7c57ea78161cc822586e57c"
Jul 26 18:43:55.666628 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:43:55.666605    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8369c4fbb455a3e6abce790821ea9d3d1668210d7c57ea78161cc822586e57c\": container with ID starting with e8369c4fbb455a3e6abce790821ea9d3d1668210d7c57ea78161cc822586e57c not found: ID does not exist" containerID="e8369c4fbb455a3e6abce790821ea9d3d1668210d7c57ea78161cc822586e57c"
Jul 26 18:43:55.666726 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.666635    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8369c4fbb455a3e6abce790821ea9d3d1668210d7c57ea78161cc822586e57c"} err="failed to get container status \"e8369c4fbb455a3e6abce790821ea9d3d1668210d7c57ea78161cc822586e57c\": rpc error: code = NotFound desc = could not find container \"e8369c4fbb455a3e6abce790821ea9d3d1668210d7c57ea78161cc822586e57c\": container with ID starting with e8369c4fbb455a3e6abce790821ea9d3d1668210d7c57ea78161cc822586e57c not found: ID does not exist"
Jul 26 18:43:55.686891 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.686866    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9"]
Jul 26 18:43:55.696214 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.696192    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-57f93c-7475b766c9-4htq9"]
Jul 26 18:43:55.838750 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.838715    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1509f49-c50f-4606-865d-08fc90687332" path="/var/lib/kubelet/pods/b1509f49-c50f-4606-865d-08fc90687332/volumes"
Jul 26 18:43:55.839084 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:55.839071    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff60c926-d113-4a1a-ac00-faa9791b6cdd" path="/var/lib/kubelet/pods/ff60c926-d113-4a1a-ac00-faa9791b6cdd/volumes"
Jul 26 18:43:59.747328 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:59.747275    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-57f93c-9tqj8"]
Jul 26 18:43:59.749513 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:59.749492    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-57f93c-9tqj8"]
Jul 26 18:43:59.838641 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:43:59.838604    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a861acd7-d8cb-4378-9fdf-019ae929eb87" path="/var/lib/kubelet/pods/a861acd7-d8cb-4378-9fdf-019ae929eb87/volumes"
Jul 26 18:44:04.135740 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:04.135712    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 18:44:04.150176 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:04.150152    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 18:44:04.486234 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:04.486145    2570 scope.go:117] "RemoveContainer" containerID="edcd37ad483c92026d2fb50d5f840f41eb22ca227b48120299aab5bf2f099e5c"
Jul 26 18:44:06.497900 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:06.497857    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6aa894-r9l7g"]
Jul 26 18:44:06.499247 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:06.499219    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff60c926-d113-4a1a-ac00-faa9791b6cdd" containerName="maas-api"
Jul 26 18:44:06.499411 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:06.499397    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff60c926-d113-4a1a-ac00-faa9791b6cdd" containerName="maas-api"
Jul 26 18:44:06.499528 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:06.499518    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a861acd7-d8cb-4378-9fdf-019ae929eb87" containerName="revoke-keys"
Jul 26 18:44:06.499614 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:06.499605    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a861acd7-d8cb-4378-9fdf-019ae929eb87" containerName="revoke-keys"
Jul 26 18:44:06.499871 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:06.499857    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff60c926-d113-4a1a-ac00-faa9791b6cdd" containerName="maas-api"
Jul 26 18:44:06.499968 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:06.499958    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a861acd7-d8cb-4378-9fdf-019ae929eb87" containerName="revoke-keys"
Jul 26 18:44:06.505628 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:06.505606    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6aa894-r9l7g"]
Jul 26 18:44:06.505729 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:06.505692    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6aa894-r9l7g"
Jul 26 18:44:06.507674 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:06.507654    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-swzk5\""
Jul 26 18:44:06.601938 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:06.601905    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e30cf33-4565-47a4-b7c3-f4d359e2c5de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-6aa894-r9l7g\" (UID: \"4e30cf33-4565-47a4-b7c3-f4d359e2c5de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6aa894-r9l7g"
Jul 26 18:44:06.703263 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:06.703226    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e30cf33-4565-47a4-b7c3-f4d359e2c5de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-6aa894-r9l7g\" (UID: \"4e30cf33-4565-47a4-b7c3-f4d359e2c5de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6aa894-r9l7g"
Jul 26 18:44:06.703881 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:06.703859    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e30cf33-4565-47a4-b7c3-f4d359e2c5de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-6aa894-r9l7g\" (UID: \"4e30cf33-4565-47a4-b7c3-f4d359e2c5de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6aa894-r9l7g"
Jul 26 18:44:06.815905 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:06.815875    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6aa894-r9l7g"
Jul 26 18:44:06.951710 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:06.951683    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6aa894-r9l7g"]
Jul 26 18:44:06.953786 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:44:06.953763    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e30cf33_4565_47a4_b7c3_f4d359e2c5de.slice/crio-eed8c6fa15ef089438393d10704797d13cf69286f1ce54fb97a24173b2b27cff WatchSource:0}: Error finding container eed8c6fa15ef089438393d10704797d13cf69286f1ce54fb97a24173b2b27cff: Status 404 returned error can't find the container with id eed8c6fa15ef089438393d10704797d13cf69286f1ce54fb97a24173b2b27cff
Jul 26 18:44:07.734228 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:07.734193    2570 generic.go:356] "Generic (PLEG): container finished" podID="4e30cf33-4565-47a4-b7c3-f4d359e2c5de" containerID="8f92381522fb7dc97fd6c16e104b2172be177c209019fe371cac628c23a57d94" exitCode=0
Jul 26 18:44:07.734603 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:07.734271    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6aa894-r9l7g" event={"ID":"4e30cf33-4565-47a4-b7c3-f4d359e2c5de","Type":"ContainerDied","Data":"8f92381522fb7dc97fd6c16e104b2172be177c209019fe371cac628c23a57d94"}
Jul 26 18:44:07.734603 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:07.734320    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6aa894-r9l7g" event={"ID":"4e30cf33-4565-47a4-b7c3-f4d359e2c5de","Type":"ContainerStarted","Data":"eed8c6fa15ef089438393d10704797d13cf69286f1ce54fb97a24173b2b27cff"}
Jul 26 18:44:08.874164 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:08.874137    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6aa894-r9l7g"
Jul 26 18:44:09.024344 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:09.024313    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e30cf33-4565-47a4-b7c3-f4d359e2c5de-maas-api-service-ca\") pod \"4e30cf33-4565-47a4-b7c3-f4d359e2c5de\" (UID: \"4e30cf33-4565-47a4-b7c3-f4d359e2c5de\") "
Jul 26 18:44:09.024717 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:09.024694    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e30cf33-4565-47a4-b7c3-f4d359e2c5de-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4e30cf33-4565-47a4-b7c3-f4d359e2c5de" (UID: "4e30cf33-4565-47a4-b7c3-f4d359e2c5de"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:44:09.125766 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:09.125731    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e30cf33-4565-47a4-b7c3-f4d359e2c5de-maas-api-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:44:09.746094 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:09.746050    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6aa894-r9l7g" event={"ID":"4e30cf33-4565-47a4-b7c3-f4d359e2c5de","Type":"ContainerDied","Data":"eed8c6fa15ef089438393d10704797d13cf69286f1ce54fb97a24173b2b27cff"}
Jul 26 18:44:09.746094 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:09.746079    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6aa894-r9l7g"
Jul 26 18:44:09.746348 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:09.746108    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eed8c6fa15ef089438393d10704797d13cf69286f1ce54fb97a24173b2b27cff"
Jul 26 18:44:16.892772 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:16.892734    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4"]
Jul 26 18:44:16.893281 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:16.892976    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4" podUID="39a258bd-2d71-45e2-b313-c4bbfc1800c6" containerName="maas-api" containerID="cri-o://737de2582e54c3f258fb2d51f2ddc5c1e47876e213879689522725137745d1ac" gracePeriod=30
Jul 26 18:44:16.921677 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:16.921647    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6aa894-29751510-kwqw6"]
Jul 26 18:44:16.923753 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:16.923731    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6aa894-29751510-kwqw6"]
Jul 26 18:44:17.146359 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:17.146284    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4"
Jul 26 18:44:17.187048 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:17.187011    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39a258bd-2d71-45e2-b313-c4bbfc1800c6-openshift-service-ca\") pod \"39a258bd-2d71-45e2-b313-c4bbfc1800c6\" (UID: \"39a258bd-2d71-45e2-b313-c4bbfc1800c6\") "
Jul 26 18:44:17.187274 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:17.187143    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w24vd\" (UniqueName: \"kubernetes.io/projected/39a258bd-2d71-45e2-b313-c4bbfc1800c6-kube-api-access-w24vd\") pod \"39a258bd-2d71-45e2-b313-c4bbfc1800c6\" (UID: \"39a258bd-2d71-45e2-b313-c4bbfc1800c6\") "
Jul 26 18:44:17.187274 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:17.187194    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39a258bd-2d71-45e2-b313-c4bbfc1800c6-maas-api-tls\") pod \"39a258bd-2d71-45e2-b313-c4bbfc1800c6\" (UID: \"39a258bd-2d71-45e2-b313-c4bbfc1800c6\") "
Jul 26 18:44:17.187414 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:17.187382    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39a258bd-2d71-45e2-b313-c4bbfc1800c6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "39a258bd-2d71-45e2-b313-c4bbfc1800c6" (UID: "39a258bd-2d71-45e2-b313-c4bbfc1800c6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:44:17.189279 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:17.189251    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39a258bd-2d71-45e2-b313-c4bbfc1800c6-kube-api-access-w24vd" (OuterVolumeSpecName: "kube-api-access-w24vd") pod "39a258bd-2d71-45e2-b313-c4bbfc1800c6" (UID: "39a258bd-2d71-45e2-b313-c4bbfc1800c6"). InnerVolumeSpecName "kube-api-access-w24vd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:44:17.189279 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:17.189264    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39a258bd-2d71-45e2-b313-c4bbfc1800c6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "39a258bd-2d71-45e2-b313-c4bbfc1800c6" (UID: "39a258bd-2d71-45e2-b313-c4bbfc1800c6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:44:17.287773 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:17.287740    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w24vd\" (UniqueName: \"kubernetes.io/projected/39a258bd-2d71-45e2-b313-c4bbfc1800c6-kube-api-access-w24vd\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:44:17.287773 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:17.287769    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39a258bd-2d71-45e2-b313-c4bbfc1800c6-maas-api-tls\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:44:17.287773 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:17.287781    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39a258bd-2d71-45e2-b313-c4bbfc1800c6-openshift-service-ca\") on node \"ip-10-0-128-45.ec2.internal\" DevicePath \"\""
Jul 26 18:44:17.796408 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:17.796376    2570 generic.go:356] "Generic (PLEG): container finished" podID="39a258bd-2d71-45e2-b313-c4bbfc1800c6" containerID="737de2582e54c3f258fb2d51f2ddc5c1e47876e213879689522725137745d1ac" exitCode=0
Jul 26 18:44:17.796603 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:17.796416    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4"
Jul 26 18:44:17.796603 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:17.796451    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4" event={"ID":"39a258bd-2d71-45e2-b313-c4bbfc1800c6","Type":"ContainerDied","Data":"737de2582e54c3f258fb2d51f2ddc5c1e47876e213879689522725137745d1ac"}
Jul 26 18:44:17.796603 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:17.796501    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4" event={"ID":"39a258bd-2d71-45e2-b313-c4bbfc1800c6","Type":"ContainerDied","Data":"d0ef616f365e7ed3f024b5813780000098c74e3822646ef279a62e85b89162ef"}
Jul 26 18:44:17.796603 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:17.796522    2570 scope.go:117] "RemoveContainer" containerID="737de2582e54c3f258fb2d51f2ddc5c1e47876e213879689522725137745d1ac"
Jul 26 18:44:17.806854 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:17.806835    2570 scope.go:117] "RemoveContainer" containerID="737de2582e54c3f258fb2d51f2ddc5c1e47876e213879689522725137745d1ac"
Jul 26 18:44:17.807159 ip-10-0-128-45 kubenswrapper[2570]: E0726 18:44:17.807099    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"737de2582e54c3f258fb2d51f2ddc5c1e47876e213879689522725137745d1ac\": container with ID starting with 737de2582e54c3f258fb2d51f2ddc5c1e47876e213879689522725137745d1ac not found: ID does not exist" containerID="737de2582e54c3f258fb2d51f2ddc5c1e47876e213879689522725137745d1ac"
Jul 26 18:44:17.807251 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:17.807184    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"737de2582e54c3f258fb2d51f2ddc5c1e47876e213879689522725137745d1ac"} err="failed to get container status \"737de2582e54c3f258fb2d51f2ddc5c1e47876e213879689522725137745d1ac\": rpc error: code = NotFound desc = could not find container \"737de2582e54c3f258fb2d51f2ddc5c1e47876e213879689522725137745d1ac\": container with ID starting with 737de2582e54c3f258fb2d51f2ddc5c1e47876e213879689522725137745d1ac not found: ID does not exist"
Jul 26 18:44:17.819273 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:17.819246    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4"]
Jul 26 18:44:17.822915 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:17.822891    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6aa894-646f558b8b-qjjx4"]
Jul 26 18:44:17.838344 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:17.838324    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39a258bd-2d71-45e2-b313-c4bbfc1800c6" path="/var/lib/kubelet/pods/39a258bd-2d71-45e2-b313-c4bbfc1800c6/volumes"
Jul 26 18:44:17.838615 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:17.838602    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9522439-711f-4d22-af3f-790fa30cea7a" path="/var/lib/kubelet/pods/a9522439-711f-4d22-af3f-790fa30cea7a/volumes"
Jul 26 18:44:21.565512 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:21.565472    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6aa894-r9l7g"]
Jul 26 18:44:21.567812 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:21.567782    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6aa894-r9l7g"]
Jul 26 18:44:21.838804 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:21.838718    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e30cf33-4565-47a4-b7c3-f4d359e2c5de" path="/var/lib/kubelet/pods/4e30cf33-4565-47a4-b7c3-f4d359e2c5de/volumes"
Jul 26 18:44:28.778650 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:28.778572    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-76ff7f7cbc-zzzx2_1e33d04d-5a1e-48b3-9fd1-c1c61cb2fdaf/authorino/0.log"
Jul 26 18:44:34.629241 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:34.629206    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-jfjsd_318aa469-e0e0-4985-8f40-01b5557ca6eb/manager/0.log"
Jul 26 18:44:34.740631 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:34.740598    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b79b7df6c-l2rp6_86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa/manager/0.log"
Jul 26 18:44:34.746548 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:34.746523    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b79b7df6c-l2rp6_86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa/copy-manifests/0.log"
Jul 26 18:44:35.351368 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:35.351335    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-664bf9fcf6-np9qj_ae329aad-5450-45d6-9c7f-82f6ff9e32d6/maas-api/0.log"
Jul 26 18:44:35.464441 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:35.464408    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29751480-gzjtm_86890ed7-3503-454a-b6e9-8805d925ff27/cleanup/0.log"
Jul 26 18:44:35.581475 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:35.581448    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29751495-8x728_23736da4-1b7c-4824-8a92-8a84a4db2a0a/cleanup/0.log"
Jul 26 18:44:35.689661 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:35.689574    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29751510-8p4tw_cc3f950f-3cf3-47ff-8aab-9709065ad09b/cleanup/0.log"
Jul 26 18:44:36.556189 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:36.556156    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn_ec4e20d8-f65b-42e6-b43b-ca57ebefe970/util/0.log"
Jul 26 18:44:36.562359 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:36.562326    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn_ec4e20d8-f65b-42e6-b43b-ca57ebefe970/pull/0.log"
Jul 26 18:44:36.568036 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:36.568007    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn_ec4e20d8-f65b-42e6-b43b-ca57ebefe970/extract/0.log"
Jul 26 18:44:36.680628 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:36.680601    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh_5f237abf-e1b2-4587-9744-49fc265cd2f6/extract/0.log"
Jul 26 18:44:36.686461 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:36.686440    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh_5f237abf-e1b2-4587-9744-49fc265cd2f6/util/0.log"
Jul 26 18:44:36.692589 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:36.692561    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh_5f237abf-e1b2-4587-9744-49fc265cd2f6/pull/0.log"
Jul 26 18:44:36.805896 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:36.805867    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9_168868e8-b9f1-483a-8914-779244f7f85e/util/0.log"
Jul 26 18:44:36.812282 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:36.812207    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9_168868e8-b9f1-483a-8914-779244f7f85e/pull/0.log"
Jul 26 18:44:36.818368 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:36.818348    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9_168868e8-b9f1-483a-8914-779244f7f85e/extract/0.log"
Jul 26 18:44:36.927692 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:36.927663    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2_c9ff4b55-0d26-4800-91ce-ed80d3468084/util/0.log"
Jul 26 18:44:36.934983 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:36.934958    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2_c9ff4b55-0d26-4800-91ce-ed80d3468084/pull/0.log"
Jul 26 18:44:36.941510 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:36.941490    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2_c9ff4b55-0d26-4800-91ce-ed80d3468084/extract/0.log"
Jul 26 18:44:37.061077 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:37.061049    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-76ff7f7cbc-zzzx2_1e33d04d-5a1e-48b3-9fd1-c1c61cb2fdaf/authorino/0.log"
Jul 26 18:44:37.391902 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:37.391874    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-trj28_6b9c00b5-4b2c-4728-adb5-fe8a23d68d0d/kuadrant-console-plugin/0.log"
Jul 26 18:44:38.566203 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:38.566172    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6855b89bd5-rrbbr_10d138c1-d703-4e3b-b6e9-261b93300cfe/kube-auth-proxy/0.log"
Jul 26 18:44:38.811465 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:38.811429    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-njrlb_608f4b82-aaca-455f-9977-c820a04aea61/payload-pre-processing/0.log"
Jul 26 18:44:38.932325 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:38.932224    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-fw8n6_22c0748d-5cd0-4d65-addb-2e9f3837a71d/payload-processing/0.log"
Jul 26 18:44:39.040743 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:39.040714    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65db774ff-4whvz_759b21a0-1d29-4328-ad81-1e6c9f0c5b66/router/0.log"
Jul 26 18:44:39.610688 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:39.610649    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-mfl8j_4efb9111-d468-40bb-a3c8-8f3504934007/main/0.log"
Jul 26 18:44:45.701450 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:45.701412    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wprln/must-gather-6f2ls"]
Jul 26 18:44:45.701845 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:45.701762    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e30cf33-4565-47a4-b7c3-f4d359e2c5de" containerName="revoke-keys"
Jul 26 18:44:45.701845 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:45.701773    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e30cf33-4565-47a4-b7c3-f4d359e2c5de" containerName="revoke-keys"
Jul 26 18:44:45.701845 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:45.701794    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39a258bd-2d71-45e2-b313-c4bbfc1800c6" containerName="maas-api"
Jul 26 18:44:45.701845 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:45.701799    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="39a258bd-2d71-45e2-b313-c4bbfc1800c6" containerName="maas-api"
Jul 26 18:44:45.701979 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:45.701854    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e30cf33-4565-47a4-b7c3-f4d359e2c5de" containerName="revoke-keys"
Jul 26 18:44:45.701979 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:45.701864    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="39a258bd-2d71-45e2-b313-c4bbfc1800c6" containerName="maas-api"
Jul 26 18:44:45.706444 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:45.706423    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wprln/must-gather-6f2ls"
Jul 26 18:44:45.708578 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:45.708556    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wprln\"/\"openshift-service-ca.crt\""
Jul 26 18:44:45.708664 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:45.708598    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wprln\"/\"kube-root-ca.crt\""
Jul 26 18:44:45.718721 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:45.718699    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wprln/must-gather-6f2ls"]
Jul 26 18:44:45.831727 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:45.831689    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b45bf2db-231f-465a-8088-cfbb96805b0b-must-gather-output\") pod \"must-gather-6f2ls\" (UID: \"b45bf2db-231f-465a-8088-cfbb96805b0b\") " pod="openshift-must-gather-wprln/must-gather-6f2ls"
Jul 26 18:44:45.831904 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:45.831760    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ml6b2\" (UniqueName: \"kubernetes.io/projected/b45bf2db-231f-465a-8088-cfbb96805b0b-kube-api-access-ml6b2\") pod \"must-gather-6f2ls\" (UID: \"b45bf2db-231f-465a-8088-cfbb96805b0b\") " pod="openshift-must-gather-wprln/must-gather-6f2ls"
Jul 26 18:44:45.932913 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:45.932881    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b45bf2db-231f-465a-8088-cfbb96805b0b-must-gather-output\") pod \"must-gather-6f2ls\" (UID: \"b45bf2db-231f-465a-8088-cfbb96805b0b\") " pod="openshift-must-gather-wprln/must-gather-6f2ls"
Jul 26 18:44:45.933105 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:45.932977    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ml6b2\" (UniqueName: \"kubernetes.io/projected/b45bf2db-231f-465a-8088-cfbb96805b0b-kube-api-access-ml6b2\") pod \"must-gather-6f2ls\" (UID: \"b45bf2db-231f-465a-8088-cfbb96805b0b\") " pod="openshift-must-gather-wprln/must-gather-6f2ls"
Jul 26 18:44:45.933331 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:45.933303    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b45bf2db-231f-465a-8088-cfbb96805b0b-must-gather-output\") pod \"must-gather-6f2ls\" (UID: \"b45bf2db-231f-465a-8088-cfbb96805b0b\") " pod="openshift-must-gather-wprln/must-gather-6f2ls"
Jul 26 18:44:45.940416 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:45.940394    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ml6b2\" (UniqueName: \"kubernetes.io/projected/b45bf2db-231f-465a-8088-cfbb96805b0b-kube-api-access-ml6b2\") pod \"must-gather-6f2ls\" (UID: \"b45bf2db-231f-465a-8088-cfbb96805b0b\") " pod="openshift-must-gather-wprln/must-gather-6f2ls"
Jul 26 18:44:46.015820 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:46.015788    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wprln/must-gather-6f2ls"
Jul 26 18:44:46.146316 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:46.146284    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wprln/must-gather-6f2ls"]
Jul 26 18:44:46.148156 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:44:46.148104    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb45bf2db_231f_465a_8088_cfbb96805b0b.slice/crio-5198d0c09b24b3233012dc0238a50f74a6c2204592dd85841ab41517d7b4f2fa WatchSource:0}: Error finding container 5198d0c09b24b3233012dc0238a50f74a6c2204592dd85841ab41517d7b4f2fa: Status 404 returned error can't find the container with id 5198d0c09b24b3233012dc0238a50f74a6c2204592dd85841ab41517d7b4f2fa
Jul 26 18:44:46.149832 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:46.149815    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 18:44:46.966272 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:46.966244    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wprln/must-gather-6f2ls" event={"ID":"b45bf2db-231f-465a-8088-cfbb96805b0b","Type":"ContainerStarted","Data":"5198d0c09b24b3233012dc0238a50f74a6c2204592dd85841ab41517d7b4f2fa"}
Jul 26 18:44:47.974861 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:47.974816    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wprln/must-gather-6f2ls" event={"ID":"b45bf2db-231f-465a-8088-cfbb96805b0b","Type":"ContainerStarted","Data":"ac4b0c3433d612e5df0b219e5dd18681e7ab92bda42a052103d67d3fb78c7057"}
Jul 26 18:44:47.975289 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:47.974878    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wprln/must-gather-6f2ls" event={"ID":"b45bf2db-231f-465a-8088-cfbb96805b0b","Type":"ContainerStarted","Data":"a7d4c6f74d9d282ae2419a119c8155d1ed470298be73d24e6360834af1afae42"}
Jul 26 18:44:47.991062 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:47.991012    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wprln/must-gather-6f2ls" podStartSLOduration=2.229858687 podStartE2EDuration="2.990992983s" totalImagesPullingTime="761.134296ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:44:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:44:47.989874922 +0000 UTC m=+3344.642065777" watchObservedRunningTime="2026-07-26 18:44:47.990992983 +0000 UTC m=+3344.643183841"
Jul 26 18:44:48.505182 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:48.505149    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-96jft_32cbd054-d7a7-4c00-be2e-012e16598a49/global-pull-secret-syncer/0.log"
Jul 26 18:44:48.702405 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:48.702365    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-khbbn_3514ccab-ab9c-46b1-bf1a-21cac46c7586/konnectivity-agent/0.log"
Jul 26 18:44:48.755924 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:48.755845    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-45.ec2.internal_e256c610577a2f8cee15b21c5c7fed95/haproxy/0.log"
Jul 26 18:44:53.302334 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:53.302303    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn_ec4e20d8-f65b-42e6-b43b-ca57ebefe970/extract/0.log"
Jul 26 18:44:53.324847 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:53.324799    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn_ec4e20d8-f65b-42e6-b43b-ca57ebefe970/util/0.log"
Jul 26 18:44:53.345695 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:53.345663    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598kthn_ec4e20d8-f65b-42e6-b43b-ca57ebefe970/pull/0.log"
Jul 26 18:44:53.372958 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:53.372927    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh_5f237abf-e1b2-4587-9744-49fc265cd2f6/extract/0.log"
Jul 26 18:44:53.392706 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:53.392652    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh_5f237abf-e1b2-4587-9744-49fc265cd2f6/util/0.log"
Jul 26 18:44:53.416691 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:53.416662    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ktrkh_5f237abf-e1b2-4587-9744-49fc265cd2f6/pull/0.log"
Jul 26 18:44:53.441010 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:53.440893    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9_168868e8-b9f1-483a-8914-779244f7f85e/extract/0.log"
Jul 26 18:44:53.461827 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:53.461796    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9_168868e8-b9f1-483a-8914-779244f7f85e/util/0.log"
Jul 26 18:44:53.484181 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:53.484149    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7nk9_168868e8-b9f1-483a-8914-779244f7f85e/pull/0.log"
Jul 26 18:44:53.516165 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:53.516136    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2_c9ff4b55-0d26-4800-91ce-ed80d3468084/extract/0.log"
Jul 26 18:44:53.537881 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:53.537855    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2_c9ff4b55-0d26-4800-91ce-ed80d3468084/util/0.log"
Jul 26 18:44:53.573458 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:53.573376    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mhfh2_c9ff4b55-0d26-4800-91ce-ed80d3468084/pull/0.log"
Jul 26 18:44:54.152136 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:54.151754    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-76ff7f7cbc-zzzx2_1e33d04d-5a1e-48b3-9fd1-c1c61cb2fdaf/authorino/0.log"
Jul 26 18:44:54.249397 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:54.249369    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-trj28_6b9c00b5-4b2c-4728-adb5-fe8a23d68d0d/kuadrant-console-plugin/0.log"
Jul 26 18:44:56.871410 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:56.871375    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-znqkg_e120f5e8-a16c-4e04-88c6-993fafdc7a40/node-exporter/0.log"
Jul 26 18:44:56.889685 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:56.889654    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-znqkg_e120f5e8-a16c-4e04-88c6-993fafdc7a40/kube-rbac-proxy/0.log"
Jul 26 18:44:56.912068 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:56.912030    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-znqkg_e120f5e8-a16c-4e04-88c6-993fafdc7a40/init-textfile/0.log"
Jul 26 18:44:57.244453 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:57.244363    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6db48c7d75-s7wz6_27d46ba6-398f-4d30-859d-b7a89e14942d/telemeter-client/0.log"
Jul 26 18:44:57.263455 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:57.263428    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6db48c7d75-s7wz6_27d46ba6-398f-4d30-859d-b7a89e14942d/reload/0.log"
Jul 26 18:44:57.282880 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:57.282843    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6db48c7d75-s7wz6_27d46ba6-398f-4d30-859d-b7a89e14942d/kube-rbac-proxy/0.log"
Jul 26 18:44:58.327136 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:58.327085    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj"]
Jul 26 18:44:58.331502 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:58.331473    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj"
Jul 26 18:44:58.334219 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:58.334187    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wprln\"/\"default-dockercfg-6tfqw\""
Jul 26 18:44:58.340486 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:58.340452    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj"]
Jul 26 18:44:58.459594 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:58.459554    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/72f0a413-381e-4e44-a409-af350d8b2b9a-proc\") pod \"perf-node-gather-daemonset-s7lbj\" (UID: \"72f0a413-381e-4e44-a409-af350d8b2b9a\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj"
Jul 26 18:44:58.459869 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:58.459841    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/72f0a413-381e-4e44-a409-af350d8b2b9a-lib-modules\") pod \"perf-node-gather-daemonset-s7lbj\" (UID: \"72f0a413-381e-4e44-a409-af350d8b2b9a\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj"
Jul 26 18:44:58.460058 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:58.460044    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f78ql\" (UniqueName: \"kubernetes.io/projected/72f0a413-381e-4e44-a409-af350d8b2b9a-kube-api-access-f78ql\") pod \"perf-node-gather-daemonset-s7lbj\" (UID: \"72f0a413-381e-4e44-a409-af350d8b2b9a\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj"
Jul 26 18:44:58.460208 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:58.460192    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/72f0a413-381e-4e44-a409-af350d8b2b9a-sys\") pod \"perf-node-gather-daemonset-s7lbj\" (UID: \"72f0a413-381e-4e44-a409-af350d8b2b9a\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj"
Jul 26 18:44:58.460384 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:58.460360    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/72f0a413-381e-4e44-a409-af350d8b2b9a-podres\") pod \"perf-node-gather-daemonset-s7lbj\" (UID: \"72f0a413-381e-4e44-a409-af350d8b2b9a\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj"
Jul 26 18:44:58.561875 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:58.561838    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/72f0a413-381e-4e44-a409-af350d8b2b9a-podres\") pod \"perf-node-gather-daemonset-s7lbj\" (UID: \"72f0a413-381e-4e44-a409-af350d8b2b9a\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj"
Jul 26 18:44:58.562127 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:58.562091    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/72f0a413-381e-4e44-a409-af350d8b2b9a-podres\") pod \"perf-node-gather-daemonset-s7lbj\" (UID: \"72f0a413-381e-4e44-a409-af350d8b2b9a\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj"
Jul 26 18:44:58.562357 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:58.562334    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/72f0a413-381e-4e44-a409-af350d8b2b9a-proc\") pod \"perf-node-gather-daemonset-s7lbj\" (UID: \"72f0a413-381e-4e44-a409-af350d8b2b9a\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj"
Jul 26 18:44:58.562457 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:58.562402    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/72f0a413-381e-4e44-a409-af350d8b2b9a-lib-modules\") pod \"perf-node-gather-daemonset-s7lbj\" (UID: \"72f0a413-381e-4e44-a409-af350d8b2b9a\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj"
Jul 26 18:44:58.562523 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:58.562464    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/72f0a413-381e-4e44-a409-af350d8b2b9a-proc\") pod \"perf-node-gather-daemonset-s7lbj\" (UID: \"72f0a413-381e-4e44-a409-af350d8b2b9a\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj"
Jul 26 18:44:58.562523 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:58.562471    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f78ql\" (UniqueName: \"kubernetes.io/projected/72f0a413-381e-4e44-a409-af350d8b2b9a-kube-api-access-f78ql\") pod \"perf-node-gather-daemonset-s7lbj\" (UID: \"72f0a413-381e-4e44-a409-af350d8b2b9a\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj"
Jul 26 18:44:58.562622 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:58.562532    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/72f0a413-381e-4e44-a409-af350d8b2b9a-sys\") pod \"perf-node-gather-daemonset-s7lbj\" (UID: \"72f0a413-381e-4e44-a409-af350d8b2b9a\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj"
Jul 26 18:44:58.562677 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:58.562653    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/72f0a413-381e-4e44-a409-af350d8b2b9a-sys\") pod \"perf-node-gather-daemonset-s7lbj\" (UID: \"72f0a413-381e-4e44-a409-af350d8b2b9a\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj"
Jul 26 18:44:58.562823 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:58.562790    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/72f0a413-381e-4e44-a409-af350d8b2b9a-lib-modules\") pod \"perf-node-gather-daemonset-s7lbj\" (UID: \"72f0a413-381e-4e44-a409-af350d8b2b9a\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj"
Jul 26 18:44:58.570870 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:58.570839    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f78ql\" (UniqueName: \"kubernetes.io/projected/72f0a413-381e-4e44-a409-af350d8b2b9a-kube-api-access-f78ql\") pod \"perf-node-gather-daemonset-s7lbj\" (UID: \"72f0a413-381e-4e44-a409-af350d8b2b9a\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj"
Jul 26 18:44:58.583552 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:58.583479    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-m4lr7_df483052-a7ce-455d-b77b-471ef05da766/networking-console-plugin/0.log"
Jul 26 18:44:58.652938 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:58.652887    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj"
Jul 26 18:44:58.837204 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:58.836170    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj"]
Jul 26 18:44:58.849369 ip-10-0-128-45 kubenswrapper[2570]: W0726 18:44:58.844667    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod72f0a413_381e_4e44_a409_af350d8b2b9a.slice/crio-32304dc0d0b4905527ac69bbc978304ff4fca834370089c3c441663da320781e WatchSource:0}: Error finding container 32304dc0d0b4905527ac69bbc978304ff4fca834370089c3c441663da320781e: Status 404 returned error can't find the container with id 32304dc0d0b4905527ac69bbc978304ff4fca834370089c3c441663da320781e
Jul 26 18:44:59.081714 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:59.081677    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj" event={"ID":"72f0a413-381e-4e44-a409-af350d8b2b9a","Type":"ContainerStarted","Data":"14a9216d020445f5675ef8a991d1e4498e85a340eacc23ed135032aeee89a3ee"}
Jul 26 18:44:59.082013 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:59.081978    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj" event={"ID":"72f0a413-381e-4e44-a409-af350d8b2b9a","Type":"ContainerStarted","Data":"32304dc0d0b4905527ac69bbc978304ff4fca834370089c3c441663da320781e"}
Jul 26 18:44:59.082958 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:59.082936    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj"
Jul 26 18:44:59.102021 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:59.101914    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj" podStartSLOduration=1.101894909 podStartE2EDuration="1.101894909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:44:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:44:59.099268952 +0000 UTC m=+3355.751459817" watchObservedRunningTime="2026-07-26 18:44:59.101894909 +0000 UTC m=+3355.754085781"
Jul 26 18:44:59.184517 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:59.184486    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/2.log"
Jul 26 18:44:59.193331 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:59.193278    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-q8vbm_801b12f8-cc05-4646-99a0-304297466b26/console-operator/3.log"
Jul 26 18:44:59.708152 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:59.708105    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-84476cb65f-d4sm9_72b366d6-ec21-49f4-b656-a07aed1f25c7/console/0.log"
Jul 26 18:44:59.747386 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:44:59.747355    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-6mtw5_2a2f2007-9f70-43a5-9481-decd359f701d/download-server/0.log"
Jul 26 18:45:00.298809 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:00.298775    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-hsdvj_51d5a4bf-5978-4bb0-81e8-3004f8519805/volume-data-source-validator/0.log"
Jul 26 18:45:01.283049 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:01.283021    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-26rxm_c07c88c4-6137-4a7e-b0ea-4e266eb50761/dns/0.log"
Jul 26 18:45:01.309764 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:01.309733    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-26rxm_c07c88c4-6137-4a7e-b0ea-4e266eb50761/kube-rbac-proxy/0.log"
Jul 26 18:45:01.462253 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:01.462216    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-974ql_1fcdab4c-350c-44c3-9bc0-a9995d33c539/dns-node-resolver/0.log"
Jul 26 18:45:02.128853 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:02.128800    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ddfsj_0557c247-c431-4eef-9de0-0b272b9e5250/node-ca/0.log"
Jul 26 18:45:03.375771 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:03.375742    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6855b89bd5-rrbbr_10d138c1-d703-4e3b-b6e9-261b93300cfe/kube-auth-proxy/0.log"
Jul 26 18:45:03.452366 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:03.452323    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751480-gzjtm"]
Jul 26 18:45:03.454748 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:03.454723    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751480-gzjtm"]
Jul 26 18:45:03.499885 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:03.499847    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-njrlb_608f4b82-aaca-455f-9977-c820a04aea61/payload-pre-processing/0.log"
Jul 26 18:45:03.540366 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:03.540330    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-fw8n6_22c0748d-5cd0-4d65-addb-2e9f3837a71d/payload-processing/0.log"
Jul 26 18:45:03.563158 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:03.563127    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65db774ff-4whvz_759b21a0-1d29-4328-ad81-1e6c9f0c5b66/router/0.log"
Jul 26 18:45:03.840389 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:03.840354    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86890ed7-3503-454a-b6e9-8805d925ff27" path="/var/lib/kubelet/pods/86890ed7-3503-454a-b6e9-8805d925ff27/volumes"
Jul 26 18:45:04.406916 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:04.406885    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c9hsm_3321b850-63be-4345-ac65-afaee093369a/serve-healthcheck-canary/0.log"
Jul 26 18:45:04.516924 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:04.516891    2570 scope.go:117] "RemoveContainer" containerID="171ae656604e663cb03dc2d17706bb67290ace0269b1814e8279957eb4f75337"
Jul 26 18:45:04.529549 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:04.529521    2570 scope.go:117] "RemoveContainer" containerID="423fb79affb977451eaace38a9364f755719b5fa510eeb47cb749e2114595a28"
Jul 26 18:45:05.163022 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:05.162911    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kdznm_28761b10-b1ee-441e-946c-bdd7ad7e9eed/kube-rbac-proxy/0.log"
Jul 26 18:45:05.182151 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:05.182099    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kdznm_28761b10-b1ee-441e-946c-bdd7ad7e9eed/exporter/0.log"
Jul 26 18:45:05.201043 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:05.201015    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kdznm_28761b10-b1ee-441e-946c-bdd7ad7e9eed/extractor/0.log"
Jul 26 18:45:06.107795 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:06.107764    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wprln/perf-node-gather-daemonset-s7lbj"
Jul 26 18:45:08.220882 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:08.220801    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-jfjsd_318aa469-e0e0-4985-8f40-01b5557ca6eb/manager/0.log"
Jul 26 18:45:08.246702 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:08.246671    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b79b7df6c-l2rp6_86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa/manager/0.log"
Jul 26 18:45:08.267449 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:08.267421    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b79b7df6c-l2rp6_86e0625a-e1d7-4cd8-9f6d-b8c0d50c50fa/copy-manifests/0.log"
Jul 26 18:45:09.133319 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:09.133274    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6855b89bd5-rrbbr_10d138c1-d703-4e3b-b6e9-261b93300cfe/kube-auth-proxy/0.log"
Jul 26 18:45:09.235007 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:09.234977    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-njrlb_608f4b82-aaca-455f-9977-c820a04aea61/payload-pre-processing/0.log"
Jul 26 18:45:09.265267 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:09.265236    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-fw8n6_22c0748d-5cd0-4d65-addb-2e9f3837a71d/payload-processing/0.log"
Jul 26 18:45:09.282005 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:09.281980    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65db774ff-4whvz_759b21a0-1d29-4328-ad81-1e6c9f0c5b66/router/0.log"
Jul 26 18:45:09.669084 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:09.669045    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-fdddb58cf-z4cng_d382754b-8e98-4863-91ec-129ea9a09ac0/manager/0.log"
Jul 26 18:45:10.947743 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:10.947701    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6855b89bd5-rrbbr_10d138c1-d703-4e3b-b6e9-261b93300cfe/kube-auth-proxy/0.log"
Jul 26 18:45:11.055843 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:11.055809    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-njrlb_608f4b82-aaca-455f-9977-c820a04aea61/payload-pre-processing/0.log"
Jul 26 18:45:11.086851 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:11.086810    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-fw8n6_22c0748d-5cd0-4d65-addb-2e9f3837a71d/payload-processing/0.log"
Jul 26 18:45:11.101033 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:11.101003    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65db774ff-4whvz_759b21a0-1d29-4328-ad81-1e6c9f0c5b66/router/0.log"
Jul 26 18:45:14.765308 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:14.765271    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc_40cb8009-e6c4-4fc2-a574-c4ecca296db4/kube-storage-version-migrator-operator/1.log"
Jul 26 18:45:14.767020 ip-10-0-128-45 kubenswrapper[2570]: I0726 18:45:14.766992    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-jnlsc_40cb8009-e6c4-4fc2-a574-c4ecca296db4/kube-storage-version-migrator-operator/0.log"